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ABSTRACT 


Modem combat operations are predominantly joint or combined arms, in which 
different forces and weapon systems come together to fight as a single entity—as a 
system composed of many different systems. For land forces, system of systems typically 
exists at battalion and above-sized forces. This thesis investigates the effects of two types 
of terrain (urban and rivers) on combat operations. Using a synthesis of various 
simulation techniques (rapid scenario generation (RSG), red teaming, experimental 
design, data analysis, and cluster and outlier analysis), 2,827 of these operations are 
simulated to understand how the individual systems perform and provide insights into the 
effects of terrain on battle outcomes. With the operational scenario requiring the 
simulation of force sizes that were the largest ever attempted (battalion and brigade for 
the urban and river crossing scenarios, respectively) in Map Aware Nonuniform 
Automata (MANA, an agent-based simulation environment), an RSG tool was developed. 
This tool allows future MANA users to easily create combat models at the systems level. 
Results indicate that both types of terrain are disadvantageous for the attacker, especially 
the urban terrain. It is found that success in the attack relies critically on the survivability 
of armor protection, specifically to be able to survive at least three good shots from 
antiarmor weapons. In addition, for both the defender and attacker, responsive 
communications was identified as a key determinant of battle outcome and a threshold of 
less than 1.5 to 2 minutes is required for communications to be effective in enabling 
effective indirect fires. 
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THESIS DISCLAIMER 


The reader is cautioned that the computer programs presented in this research may 
not have been exercised for all cases of interest. While every effort has been made, within 
the time available, to ensure that the programs are free of computational and logical 
errors, they cannot be considered validated. Any application of these programs without 
additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


This thesis proposes a methodology to study a combat system of systems. It aims 
to provide decision makers with insights at the systems level on prioritizing scarce 
resources to support the development of new capabilities. A typical system of systems 
consists of the following elements: Fire, Maneuver, Mobility/Counter-Mobility, 
Command, Control, and Communications (C3), Intelligence (I), and Combat Service 
Support (CSS). For land forces, a system of systems typically exists in battalion and 
above-sized forces. 

In addition, this thesis provides insights into how the various sensors, 
communications, weapons systems, and forces interact in determining the battle outcome 
for both the defender and attacker in two types of terrain, namely urban areas and rivers. 
Both of these pose unique challenges for the attacker and defender. With the increasing 
urbanization of the world, land forces are expected to be able to conduct combat 
operations within an urban environment. Urban terrain has the characteristic of being 
multidimensional, with abundant cover and concealment, along with axes that are 
dominated by observation and fire from structures along the axes. Moreover, the presence 
of civilians in the urban environment adds another layer of complexity, where Rules of 
Engagement (ROE) become important. Rivers have been conventionally a natural line of 
defense that defending forces exploit to stop the attacking force from advancing. A river 
crossing operation is a race between the crossing force and the opposing force to mass 
combat power on the far shore. The longer the crossing force takes to cross, the less 
likely it will succeed—as the opposing force will defeat the elements split by the river. 
This thesis is guided by two questions to provide insight into the operational issues to 
be answered: 

• How effective are communications, sensors, weapons, and platforms with 
regard to enabling the combat system of systems to complete its mission 
under the two types of scenarios? 

• How do the characteristics of the various terrain (urban and rivers) impact 
the combat forces’ performance? 
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The scenarios have been formulated for the study of combat system-of-systems in 
these two types of terrain: 

• Urban Operations (UO): In this scenario, the attacker is a mechanized 
infantry battalion and the defender is a reinforced company of mechanized 
infantry. The size of the area of operations is 5km by 5km. 

• River Crossing Operations (RCO): This scenario consists of an armor 
brigade minus attempting to secure a breakout point at the far bank of a 
river. The river is defended by a battalion minus of infantry, reinforced 
with a company of motorized infantry as reserves. The size of the area of 
operations is 10km by 10km. 

The terrain maps of the two scenarios are shown in Figure 1. 



Figure 1. Terrain Maps of the Two Scenarios 


In this thesis, a synthesis of different simulation and analysis techniques (Rapid 
Scenario Generation [RSG], Manual and Automated Red Teaming [ART], Experimental 
Design and Data Farming, Data Analysis, Cluster and Outlier Analysis for Data Mining 
[COADM]) were employed to study this complex system of systems. One of the key 
contributions of this thesis is the development of an RSG tool for the building of very 
large models in Map Aware Nonuniform Automata (MANA). MANA is a much used 
stochastic, agent-based combat simulation environment. 
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Typical agent-based models study up to, at most, company-sized forces; however, 
the operational scenarios that were studied in this thesis require force sizes of battalion 
and above. In addition, the complexity of building the combat model increases 
exponentially as we increase the force size (e.g., company to battalion). Hence, to rapidly 
scale up the various components of the model (e.g., infantry squads, armor, and artillery 
detachments) to a battalion and above-sized force, an RSG tool was required. This tool 
can be used by other MANA users to rapidly scale up their scenario as needed. It allows 
the rapid building of the model up to any desired size (subject to the limitations of 
MANA) once the basic building blocks of squad characteristics, communication links, 
and waypoints are created. In addition, it also enables automation of the design of 
experiments process. A total of 2,827 of these operations (including both urban and river 
crossing) were simulated using high-performance computing clusters to generate the data 
required for the research. The analysis of these data provided answers to the questions 
posed by this thesis and provided additional insights as well. 

With regard to the effects of terrain on performance, results showed that the 
probability of mission success for the attacker is 26% in the urban scenario and 52% in 
the river crossing operations, when modeled using a wide spectrum of capabilities and 
terrain conditions. 

We can see that these two types of operations are indeed challenging for the 
attacker, with only about a 50% chance of success at best. In addition, we see that the 
urban terrain scenario is more treacherous, compared to that of the river crossing 
operation, as it is half as likely to succeed. Moreover, both scenarios show that the 
defender needs to exploit the characteristics of the terrain to his advantage. In urban 
operations, the cover afforded by the terrain is the key factor, whereas in river crossing 
operations, concealment afforded by the terrain is more important. This provides us with 
an important conclusion—that with good concealment afforded by vegetation and 
overhead foliage in the river crossing scenario, it would be advantageous for the attacker 
to have sensors that are able to “look” beneath the foliage. 

With regard to the effectiveness of communications, sensors, weapons, and 
platforms, the analysis shows the following: 
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• Attacker: The survivability of armor is critical for success, specifically, 
armor protection on vehicles should be able to survive at least three good 
shots (deflect or absorb). This produces a higher casualties for the 
defender. 

• Attacker: For targeting of the defender’s reserves to be effective, the 
latency of the sensors in transmitting information back to the shooter has 
to be less than 1.5 to 2 minutes. If the sensor-shooter latency is higher than 
that threshold, then it becomes important for the closer range antitank 
weapons to be effective to take out the defender’s reserves. This finding 
also provided an additional insight in that where the sensor-shooter chain 
has high latency, the attacker should advance more slowly. This allows the 
reserves to be deployed in position and then “softened” with fires before 
the attacker commences capture of the position. 

• Defender: Communications latency of less than 1.5 to 2 minutes is the 
identified threshold for reducing defender’s casualties. Expanding upon 
this point, we can conclude that if the attacker is able to jam the defender’s 
communications effectively prior and during the attack, he would be able 
to blunt one of the defender’s few sources of strength. 

In addition to insights on the operational scenarios, the thesis also revealed the 
limits of the MANA model. It was discovered that MANA can handle up to a maximum 
of around 1,000-plus agents and 2,000-plus communication links. This is the size of the 
larger river crossing scenario, which involves a brigade-minus force versus a 
battalion-minus force. A previously-built larger scenario (brigade versus battalion) 
caused MANA to run out of memory; hence, the scenario was subsequently trimmed 
down. In addition, the run time for that scenario was around 24 hours per run. Therefore, 
for a single replicate, it consumes 6,168 hours of computer processing unit (CPU) time, 
which translates to approximately one week to produce a single replicate. 

This thesis demonstrated the synthesis of various simulation techniques to provide 
answers to operational issues from a system of systems perspective. The analytic results 
and tools developed can be used to assist decision makers in the development of policies, 
concept of operations for new systems, and also to evaluate tactics, techniques, and 
procedures (TTPs). 
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I. INTRODUCTION 


A. OVERVIEW 

It is possible for everybody to buy ships, tanks or airplanes, but the key is 
how you operate them together in a concept that best suits your 
operational needs and environment... 


RADM(NS) Teo Chee Hean 
Minister of Defense, Singapore 
Opening Address at Singapore Armed Forces (SAF) Centre for Military 

Experimentation (SCME) 29 Dec 2003 
(Defense Science Technology Agency, 2003) 

Modern warfare is characterized by the use of combined arms, where various 
types of forces (e.g., combat, combat support, and combat service support) with different 
weapon systems and equipment come together to fight as a single entity — as a combat 
system of systems. Figure 2 shows how the various elements of a system of 
systems interact. 



C 3 I 

Maneuver Mobility 

ttH 

Combat Service 
Support 

Figure 2. System of Systems 

A typical system of systems consists of the following components: 

• Fire: This is ultimately what a combat force delivers onto its opponents to 
attrite them and win the battle. Fire can come in two forms—direct fire 
and indirect fire. Direct fire is defined as fires that are delivered with a 
straight trajectory, with a direct line-of-sight to the target. These include 
small arms fire (e.g., M16, AK47, etc.), tank rounds, and antitank weapons. 
Indirect fire is delivered with weapons that do not require direct aim at the 
target (e.g., artillery guns, precision guided missiles, and close air strikes). 
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• Maneuver: These are the forces that engage in direct combat with the 
opponent. They include forces like infantry and armor. The forces in this 
component maneuver to deliver fire on the enemy so as to be able to 
suppress the enemy and maneuver. 

• Mobility: Opposing sides will seek to impede the ability of their enemy 
by exploiting the natural features of the terrain and by creating obstacles to 
slow the advance of the opponent. This will disrupt the opponent’s ability 
to mass his combat power and provide opportunities for fire to be 
delivered to attrite him. In this aspect, the combat engineers provide the 
capability for enhancing the mobility of our own force by breaching 
obstacles and emplacing obstacles to impede the mobility of the enemy. 

• Command, Control, and Communications, Intelligence (C3I): This is 
the component that synthesizes the elements of fire, maneuver, and 
mobility together. It is here where the information from sensors emplaced 
within the battlefield is fused and disseminated to the combat units 
through the various communication links. 

• Combat Service Support (CSS): The combat force needs to be 
adequately resupplied between battles. CSS comes in five main 
components: manpower, material, maintenance, movement, and medical. 
In this study, CSS is excluded as each of the scenarios consists of only one 
major battle. 

B. BACKGROUND AND MOTIVATION 

1. Urban Terrain - A Multidimensional Battlefield 

Army forces will likely conduct operations in and around urban areas— 
not as a matter of fate, but as a deliberate choice linked to national 
objectives and strategy and at a time, place, and method of the 
commander’s choosing. 


FM 3-06, Urban Operations 
(Headquarters, Department of the 
Army, 2003) 

As a consequence of global urbanization, a trend of migration from rural to urban 
areas is occurring throughout the world. Coupled with the exponential increase in world 
population, massive urban areas hold the centers of population as people move from rural 
to urban areas. It is estimated that almost half the world’s population resides in cities and 
urban areas (Headquarters, Department of the Army, 2003). 
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The urban terrain consists of natural and man-made features, with man-made 
features being the major part. Buildings, streets, and other infrastructure within the urban 
terrain have very different patterns, shapes, and sizes. The infinitely many ways in which 
these factors can interact with one another make it difficult to describe a “typical” urban 
area. Figure 3 shows the complexity of the urban battlefield. 



(From Headquarters, Department of the Army, 2003) 


An urban area may appear relatively insignificant on a map as compared to the 
surrounding countryside. The truth is, in fact, the reverse, where the size and extent of the 
urban battlefield is many times that of a similarly-sized piece of the natural terrain. The 
sheer volume and density, as a result of the complex urban geometry and dimensions, can 
make urban operations highly resource-intensive in terms of time, manpower, 
and material. 
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2. 


Rivers - A Natural Defense Line 


A river crossing is a race between the crossing force and the enemy to 
mass combat power on the far shore. The longer the force takes to cross, 
the less likely it will succeed, as the enemy will defeat, in detail, the 
elements split by the river. Speed is so important to crossing success that 
extraordinary measures are justified to maintain it. 

FM 90-13, River Crossing Operations 
(Headquarters, Department of the Army, 1998) 

The purpose of any river crossing operation is to project combat power across a 
water obstacle to accomplish a mission. A river crossing operation is unique in that it 
requires detailed and careful planning. It also requires well-orchestrated, combined arms 
operations to ensure a successful crossing of the river. There are four major phases in a 
river crossing operation (Headquarters, Department of the Army, 1998). 

• Advance to the River (Phase I): This requires the crossing force to 
attack to capture and secure near-shore terrain, such as favourable road 
networks and crossing sites. 

• Assault Across the River (Phase II): The purpose of this phase is to 
rapidly project combat power on the far shore to eliminate the enemy’s 
direct fire onto the crossing sites and secure terrain for attack positions. 
This will facilitate the crossing of follow-on forces for the 
subsequent phases. 

• Advance From the Exit Bank (Phase III): Once the follow-on forces 
have crossed the river, elimination of any remaining direct or indirect fire 
from the crossing area will be carried out in this phase. 

• Secure the Bridgehead (Phase IV): The bridgehead must be defendable 
and large enough to accommodate the reorganization of forces that will 
break out to continue offensive combat operations beyond the bridgehead 
line. This includes securing of construction points created by the terrain to 
hold off any counterattack by the incumbent. 

Throughout history, armies have crossed rivers to pursue or retreat from enemy 
forces. The ancient Persian Army built bridges during their invasion of Greece in the fifth 
century BC. In 480 BC, the army of Xerxes commenced preparations to cross the 
Hellespont, the narrow strait in Turkey that separates Asia from Europe. This was a 
colossal challenge even by today’s standards, for the bridge had to be over a mile long. 
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Two attempts were made to construct the bridge, as the boat bridge was broken up by 
strong winds and waves in the first attempt. Eventually, the entire Persian Army (said to 
number a million men) crossed it. 

More recently, in World War II, was the crossing of the Rhine River, which 
represented Germany’s resolve at holding off Allied forces on the Western front. The 
successful crossing of the Rhine River was one of the key turning points in the battle 
for Germany. Figure 4 shows the attack plan of the Allied Forces. 
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Figure 4. Crossing of the Rhine River 
(From Allied Crossings of the Rhine North of the Ruhr) 


Operation Plunder was carried out on the night of 23 March 1945. It involved the 
crossing of the Rhine River at Rees, Wesel, and south of the Lippe Canal by the British 
Second Army, and the U.S. Ninth Army. The XVIII U.S. Airborne Corps, consisting of 
the British 6 th Airborne Division and the U.S. 17 th Airborne Division, conducted 
Operation Varsity to insert forces across the Rhine River to secure critical terrain ahead 
of the main force so as to maintain the momentum of the advance. 
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3. 


Motivation 


In designing new force structures to meet current and future operational 
requirements, a systems perspective of the operational environment (Joint Chiefs of Staff, 
2006) is fundamental. A systems perspective considers more than just an adversary’s 
military capabilities; it strives to provide an understanding of the interrelated systems of 
both its own forces and the enemy interacting with the operational environment relevant 
to an operation. 

Studying battalion-sized and above forces can be especially difficult in live 
environments, as they require large numbers of personnel, including control personnel, 
and safety setup. In addition, “noise” present in the data set cannot be easily isolated. 
Although constructive war-gaming exercises can cut down on the number of live troops 
participating, it still requires a significant involvement of military planners to “man” the 
simulation and exercise control personnel to take data for after-action reviews (AARs). 
Moreover, the number of data points and factors that can be varied for subsequent study 
is severely limited by time and resources. Very often, only a handful of data points can be 
extracted, which does not provide for a robust and holistic analysis of the entire combat 
force. In order to better understand how the individual systems interact with each other to 
produce a battle outcome, the individual factors of the various systems have to be varied 
systematically to ensure that one can carry out an analysis. 

With agent-based simulations, the study can be “unmanned” and allow for many 
more factors to be varied for an in-depth study to be carried out. This does not imply that 
agent-based simulations are a replacement for war-gaming and live exercises. 
Agent-based simulations can provide an initial understanding of the entire combat system 
and allow us to focus on key operational issues that matter, before carrying out live 
exercises. This is especially useful during the design stage of new force structures, where 
the actual systems are not even present. 

In addition, without highly automated tools to assist in the generation of the 
model and extraction of data for analysis, the modeling of large-scale combat scenarios 
can be time-consuming and tedious. Manual generation of models and analysis can be 
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prone to error as a result of analyst fatigue. This may result in a less than robust analysis 
of the problem at hand. Hence, RSG and analytic tools need to be developed so that such 
complex systems can be modeled and analyzed with ease. 

C. RESEARCH QUESTIONS 

The aim of this thesis is to analyse the system-of-systems in an urban and river 
crossing operational scenario. The following questions guide this research: 

• How effective are communications, sensors, weapons, and platforms with 
regard to enabling the combat system-of-systems to complete its mission 
under the two types of scenarios? 

• How do the characteristics of the various terrain (e.g, urban and rivers) 
impact the performance of the combat forces? 

D. BENEFITS OF THE STUDY 

This study demonstrates the tools and framework that can be used for analyzing a 
system-of-systems, not limited just to land forces, but possibly including air and sea 
forces as well. In addition, this study provides insights into how the capabilities of the 
individual combat systems (tanks, rifles, infantry fighting vehicles) influence the final 
battle outcome. Ultimately, this thesis provides an understanding of the challenges and 
limitations of operating in the two unique terrain types. 

E. RESEARCH METHODOLOGY 

The tool used in this study is an agent-based distillation. It is a type of computer 
simulation that models only the salient features of the real world and not every possible 
characteristic (Cioppa, Lucas, & Sanchez, 2004). The tool used is Map Aware 
Nonuniform Automata (MANA), a product developed by New Zealand’s Defense 
Technology Agency (DTA). Scenarios for both terrain features (urban and rivers) are 
then developed for experimentation in MANA. 

Data farming then identifies the previously undetermined characteristics and 
situations that develop during the simulation (Cioppa, Lucas, & Sanchez, 2004). 
Statistical analysis and other analytic techniques identify the importance of the 
interactions between variables and lead to an understanding of the data. 
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In order to identify a scenario that is most threatening to the BLUE forces, the 
Automated Red Teaming (ART) (Choo, Chua, & Tay, 2007) technique is explored to 
search for possible RED deployment of reserve forces that can tilt the battle in 
RED’s favor. 
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II. SCENARIO AND MODEL DEVELOPMENT 


A. INTRODUCTION 

In order to capture the essence of how a system-of-systems conducts operations in 
both urban and river terrain, it is essential that robust and realistic scenarios are properly 
developed. In this chapter, the structure of the forces, and a brief description of the 
scenario, is covered. In addition, an overview of the MANA simulation tool that is used 
to model the combat scenarios is provided. Lastly, this chapter describes in detail how the 
simulation model behaves. Both scenarios were validated by Colonel Edward Lesnowiz, 
United Sates Marine Corps (Ret.) to confirm that they are operationally realistic and 
tactically sound. 

B. SCENARIO 

1. Urban Operations (UO) Scenario 
a. Terrain Model 

The terrain model for the urban terrain used in this scenario was developed 
by the Defense Science Organization (DSO), Singapore, in a UO study that was 
presented during the International Data Farming Workshop 15 (Team 7 - Applying 
Automated Red Teaming in an Urbans Operations Scenario, 2007). It is a map size of 
5km by 5km. The map is as depicted in Figure 5. 



Figure 5. Scenario Map of the Urban Operations Model (From: Team 7 - Applying 
Automated Red Teaming in an Urbans Operations Scenario, 2007) 
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b. Force Structure 

The attacker modeled in the UO scenario is a mechanized infantry (MI) 
battalion, structured as follows: 
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Figure 6. Structure of Attacking Force in an Urban Operations Scenario 


The defender modeled in the UO scenario is a reinforced mechanized 
infantry (MI) company deployed in the area of operations (AO). 



Figure 7. Structure of Defending Force in an Urban Operations Scenario 


10 









c. Scenario 

Defender’s (RED) Basic Deployment: At the security area, obstacles 
such as mine clusters, improvised explosive devices (IEDs), and roadblocks are 
established to cause maximum delay and to disrupt to the attacker’s advance to allow 
more time for the defender’s indirect fire to attrite the attacker’s forces. The defender 
deploys a section and a mechanized infantry platoon minus along each axis. The tank 
platoon is organized into two teams of two tanks each to reinforce the eastern and central 
axes. The defender holds a reserve of a mechanized infantry company minus at the rear of 
the area of operations. The reserves are deployed in two echelons, each consisting of a 
mechanized infantry platoon and a tank team. Each of the reserves will immediately 
reinforce the forward positions where the defenses are being breached. A pictorial 
representation of the basic deployment and deployment of reserves is shown in Figure 8. 


Resl 



Figure 8. Urban Operations Model Defender Operations Plan(Best Viewed in Color) 
(After: Team 7 - Applying Automated Red Teaming in an Urbans Operations Scenario, 

2007) 
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Attacker’s (BLUE) Concept of Operations: The attacker’s mission is to 
open at least one axis through the AO. A platoon is first inserted to the rear to establish a 
blocking position to stop the defender’s reserves from reinforcing the attacked frontal 
positions. The self-propelled howitzers (SPHs) are held at the rear to provide support fire 
to the advancing troops. The attacker then advances along two main axes, each with a 
task force consisting of one mechanized infantry company and one engineer section each, 
holding a task force in reserve. The reserve task force is employed to rush through 
whichever axis is opened successfully. Figure 9 shows the operations plan of the attacker. 



Figure 9. Urban Operations Model Attacker’s Operations Plan(Best Viewed in 
Color) (After: Team 7 - Applying Automated Red Teaming in an Urbans Operations 

Scenario, 2007) 
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2. River Crossing Operation (RCO) Scenario 
a. Terrain Model 

The terrain selected for this scenario is based upon the World War II 
crossing of the Rhine River at Rees. The map is extracted from Google Earth and 
modeled using photo editing software. In Figure 10, we see the image on the left 
as the map extracted from Google Earth and on the right is the one that has been 
rendered for use by the simulation model. 



Figure 10. River Crossing Operation Scenario Map (Best Viewed in Color) 

(From: Google Earth, 2008) 


b. Force Structure 

The attacker in the RCO scenario is a mechanized infantry brigade 
reinforced with a bridge company to support the initial crossing and a field 
artillery battalion to provide additional fire support to the crossing effort. The 
structure of the mechanized infantry(MI) brigade is shown in Figure 11. 
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Figure 11. River Crossing Operation Model Attacker Force Structure 


The structure of the bridge company and the field artillery battalion is 
shown in Figure 12. 
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Figure 12. River Crossing Operation Model Attacker Support Forces Structure 


The defender forces that are modeled consist of a reinforced infantry battalion 
with the structure shown in Figure 13. 
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Figure 13. River Crossing Operation Model Defender Force Structure 


As a result of computer and software memory limitations, however, the 
maximum size force that can be modeled is an armored brigade minus (two battalions) 
against an infantry battalion minus (two companies). 

c. Scenario 

Defender (RED) Basic Deployment: RED will deploy their forces with 
two companies to the front and one company to the rear, holding a motorized infantry 
platoon as the battalion reserve, with an additional armored infantry company to be 
reinforced by the parent brigade if required. At the forward of the main defense area 
along the river, a platoon will be deployed to dominate each of the crossing sites. 
Obstacles are manned and kept open along the key reinforcement routes. Upon 
commencement of attack on the forward positions, the battalion reserve will be launched 
to reinforce the positions that are being attacked. The company-sized brigade reserve will 
be launched upon commencement of attack on the forward positions. This defense is 
supported by two batteries of artillery equipped with counterbattery radar. The schematic 
of how the defender is deployed is shown in Figure 14 (note that the greyed elements 
indicate the forces that are not modeled as a result of model limitations in MAN A). 
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Figure 14. River Crossing Operation Model Defender’s Operations Plan 
(Best Viewed in Color) (After: Google Earth, 2008) 


Attacker (BLUE) Concept of Operations: The attacker will insert a 
company-sized force to the rear of the area of operations to attrite the reinforcements and 
prevent them from reinforcing the frontal positions. With the support of heavy artillery 
fire from a division artillery battalion to suppress the frontal objectives, as well as 
provide counterbattery fire against the defender’s artillery, the attack commences with an 
assault crossing of an armored battalion across the Rhine River. This is followed by the 
capture of the objectives directly dominating the crossing sites. Upon capture of the 
frontal objectives, the rest of the brigade, consisting of two battalions, is ferried across 
the river to secure the bridgehead to the depth of the area of operations. Figure 15 shows 
the operations plan of the attacker. 
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Figure 15. River Crossing Operation Model Attacker’s Operations Plan 
(Best Viewed in Color) (After: Google Earth, 2008) 


C. THE MANA COMBAT SIMULATION TOOL 

Having described the scenarios, this section describes the combat simulation tool. 
The tool used is an agent-based simulation tool called MANA. It is selected as the 
modeling tool best suited for this research. This section briefly covers how the tool 
works. 


1. MANA Characteristics 

MANA is in a general class of models called agent-based models (ABMs). ABMs 
have the characteristic of containing entities that are controlled by decision-making 
algorithms. MANA was built by New Zealand’s DTA for research into the complex and 
chaotic nature of real-world combat situations and to provide users with insights into 
those same situations. It employs the use of individual entities that are able to make 
decisions based upon certain rule sets that are assigned to them to explore the essence of 
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a given operational issue (McIntosh, Galligan, Anderson, & Lauren, 2007). This 
independent decision-making ability is possible with the provision of individual situation 
awareness maps as well as the agent’s personality. Its primary use is as a “distillation” 
tool; that is, to create a bottom-up abstraction of a scenario that captures just the essence 
of a situation, and leaves out details that are not essential. 

D. CHARACTERISTICS OF THE SIMULATION MODEL 

The aim of this section is to describe the characteristics of the MANA models 
created for this research. The goals of the simulation, terrain and scale, and enemy and 
friendly forces are discussed here. In addition, the sources of data and assumptions are 
also addressed. A more detailed breakdown of the behavioral characteristics and 
capabilities of all the forces can be found in Appendix B. 

1. Simulation Goal 

The scenarios developed are designed to provide a better understanding of the 
challenges of operating in the two types of terrain and to gain insights into how the 
myriad of factors in a combat situation interact with one another. In this simulation study, 
the factors that are varied are capability and environment focused. That is, factors which 
relate to force and equipment effectiveness (e.g., probability of hits, speed, armor) and 
also to the environment in which they operate in (e.g., cover and concealment, civilian 
density). These factors are varied using design of experiments techniques and explored 
over large ranges to determine how they interact and to determine the factors that are 
important and at what levels. 

2. Terrain and Scale 
a. Scale 

MANA is essentially a time-step model that requires the user to match the 
real-world scales to those used in the simulation. In this simulation, the time scale is 
coupled one-to-one with that of the real world. That is, one time step is equivalent to one 
second. The scale of the terrain maps for the two scenarios is shown in Table 1. 
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Scenario 

Real-World 
Size (km) 

MANA Size 
(pixels) 

Scale 

Urban Operations 

5x5 

500 x 500 

1 pixel: 10m 

River Crossing 

10x10 

1000 x 1000 

1 pixel: 10m 


Table 1. Scale of Terrain Maps in the UO and RCO scenarios 


b. Terrain Map 

MANA allows for various types of terrain to be modeled, which includes 
walls, buildings, bushes, and water. Three attributes of cover, concealment, and going of 
various types of terrain are modeled. These attributes take on values between 
0 and 1 and are described as follows: 

• Cover relates to the amount of protection that an individual agent receives 
when he is being fired upon; the higher this attribute is, the less likely the 
agent will be hit while being shot at by an enemy agent. 

• Concealment relates to the amount of camouflage the terrain offer agents 
which reduces their detectability by enemy agents. 

• Going describes the mobility offered by the terrain and relates to how fast 
an individual agent can move through the terrain. By setting this to zero, it 
becomes impassable terrain for the agent and this can be set for terrain 
types like walls and rivers. 

The terrains modeled in the two scenarios are listed in Table 2. 


S/N 

Scenario 

Terrain 

Going 

Cover 

Concealment 

1 . 

Urban 

Operations 

Industrial 

0.10 

0.20 

0.50 

Building 

0.10 

0.90 

0.90 

Road 

1.00 

0.00 

0.00 

2. 

River Crossing 

River 

0.00 

0.00 

0.00 

Light Bush 

0.50 

0.10 

0.30 

Buildings 

0.10 

0.90 

0.90 

Urban 

0.20 

0.60 

0.70 

Cleared Land 

1.00 

0.00 

0.00 

Road 

0.80 

0.00 

0.00 

Highway 

1.00 

0.00 

0.00 

Crossing Sites 

0.50 

0.00 

0.00 


Table 2. Terrain Information (Going, Cover, and Concealment) 
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The terrain maps are shown in Figure 16. 



Figure 16. Terrain Maps (Best Viewed in Color) (After: Google Earth, 2008 and 
Team 7 - Applying Automated Red Teaming in an Urbans Operations Scenario, 2007) 


c. Elevation Map 

To provide for a three-dimensional battlefield where different elevations 
affect forces’ line-of-sight, the elevation of the battlefield can also be specified via a 
greyscale bitmap file of the terrain map. The darker shades indicate those areas that are of 
lower elevation, whereas those that are of lighter shades correspond to that of higher 
elevations. The elevation maps of both the river crossing operation scenario and that of 
the urban operations scenario are shown in Figure 17. 



River Crossing Scenario Elevation Map Urban Ops Scenario Elevation Map 


Figure 17. Elevation Maps (After: Google Earth, 2008 and Team 7 - Applying 
Automated Red Teaming in an Urbans Operations Scenario, 2007) 
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The elevation that is modeled in these two scenarios predominantly 
consists of buildings and urban areas. 

3. Classes of Agents 

In both models, there are three classes of agents: obstacles, human, and vehicular 
agents. They are given threat classifications of I, II, and III, respectively. Obstacle agents 
model the effects of obstacles on the battlefield, which is to stop the opposing force from 
advancing. The obstacles have no effect on human agents since in reality obstacles are 
not effective in stopping infantry troops, and they are primarily employed to stop 
vehicles. Human agents can consist of civilians, infantry troops, and also mechanized 
infantry troops. Depending upon their type, they may or may not carry weapons. 
Vehicular agents are a broad class of agents consisting of armored vehicles, artillery, 
rafts, etc. These agents can have the ability to carry passengers. However, as a result of 
limitations in MANA, the agents will only be able to embuss and de-embuss the 
vehicular agent once. 

4. Generic Behavior of Forces 

This section describes the general characteristics of the forces that are modeled. In 
terms of rules of engagement (ROE), they will not engage any forces if it is likely to 
result in casualties to civilians or neutrals. With the exception of infantry agents, all 
agents who encounter an enemy obstacle agent will be stopped. This is achieved using 
the refuel option, where an agent that is being refueled by the enemy obstacle agent will 
not be able to move until the obstacle agent is cleared. In addition, all forces will slow 
down when they come into contact with an enemy force, either visually or when they are 
being shot at. 


5. Modeling of Weapons 

In order to reflect a more accurate reality that the closer the target the higher the 
probability of hit, a simple triangular range-hit probability distribution is employed. At 
the maximum effective range of the weapon, the probability of hit is taken as zero. In 
addition, if the weapon has a minimum distance for engagement (to account for arming 
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distance of rounds), the hit probability will be zero until the minimum effective range, 
where a probability of hit is assigned. The model is shown in Figure 18. 



6. Sources, Abstractions, and Assumptions 

The probabilities of hits and ranges of weapons modeled are referenced from 
(Sulewski, 2005). In that study, an extensive research on the characteristics and 
capabilities of many weapons was carried out. This included weapons from small arms to 
antitank weapons to artillery weapons. 

In both of the scenarios, CSS (which covers manpower replacement), material 
resupply (fuel, food, water, etc.), maintenance issues (e.g., failure of equipment), and 
medical evacuation are not considered. One of the reasons is that CSS becomes critical 
when there are multiple sequential missions within the scenario, where resupply becomes 
important in bringing up the readiness level of the force for the next mission. There is 
only one mission to be completed in both scenarios; hence, CSS is not modeled in this 
thesis. 


7. Bugs 

To model the effect of inaccurate information, the contact position uncertainty 
parameter was explored. The intended effect was to introduce an uncertainty in the 
information that will be transmitted among friendly forces. This was done to understand 
the impact on the MOEs when accurate communication cannot be carried out on the 
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battlefield. One of the key effects was to see if accurate coordinates calling for artillery 
fire matters and, if so, by how much. However, with the use of this parameter problems 
may occur when the enemy unit to be fired upon is located close the edge of the 
battlefield. Therefore, a position uncertainty may cause a coordinate that is outside of the 
defined battlefield grids to be transmitted to the shooter. This will unwittingly cause 
MANA to crash. 

E. RAPID SCENARIO GENERATION (RSG) TOOL 

1. Motivation 

One of the key challenges of analyzing system of systems is in the building of the 
model and setting it up for analysis. Within a system-of-systems, there are many force 
types, each with its own characteristics (weapons, behavior), mission (in terms of 
waypoints), and communication links. These characteristics are repeated in one way or 
another across the entire system. In order to be able to generate the large numbers of 
squads of agents, as well as the numerous communications links, easily within the 
system, an RSG tool is required. 

2. Concept of Tool 

A particular agent or squad has three unique characteristics that define them: 
generic force type, waypoints, and communication links. Each of these characteristics 
will be described in detail. 

a. Generic Force Type 

This describes the type of agent or squad it belongs to (e.g., mechanized 
infantry, tank, or artillery). Each of these force types have their own behavioral 
characteristics in accordance to its known TTPs, in which case, we will refer to them as 
the personality within the context of MANA. In addition, each force type will also have 
their individual weapon and sensor assignments that differ from the rest. For instance, the 
primary weapon that an artillery unit has is very different from that of an infantry soldier. 
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b. Waypoints 

The various force types can come together to form a unit. This unit will 
have routes (or waypoints in MANA) in the given AO that it has to take to complete a 
given mission. These routes will then have to be taken by the various force types that 
come under the command of the unit. 

c. Communications Links 

In order for the agents to pass information among the various force types 
within a unit, communications links have to be set up between them. Generally, the 
communications links can be in the form of two-way, where the agents communicate, or 
one-way, where the agents just use the situational awareness (SA) information relayed to 
them to carry out the mission. Each of these communications links have their individual 
characteristics, such as range of communications, time in which information takes to be 
passed through, and the type of information that is relayed (positions of enemy, own, and 
neutral forces). 

The combination of these three characteristics produces a unique squad 
with its own force type, waypoints corresponding to that of its parent unit, and how 
information is relayed between itself and the other squads. 

3. Implementation of Rapid Scenario Generation (RSG) 

The RSG tool is created in Excel and is programmed using Visual Basic for 
Applications (VBA). Excel is chosen as the tool of choice because of its ease in carrying 
out “copy-and-paste” operations. This is especially helpful where multiple units can be 
easily replicated by simply copying and pasting the particular rows and columns. 
Exploiting the characteristics of the MANA model file, which is in the Extensible 
Markup Language (XML), this tool extracts the characteristics required from each of the 
squads modeled, such as communications links, waypoints, or squad type, in the form of 
text and then reassembles all the lines of text back into one coherent scenario. The 
process for generating a scenario is as follows. 
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• Scenario files are created separately each for generic force types, 
waypoints, and communication links in MANA. The squads in each of 
these scenarios are then saved out and imported into the Excel RSG tool. 
The user will have to create a base scenario as well, one that is empty of 
squads. This base scenario file will specify the terrain maps, the terrain 
type, output file settings, etc. All generated squads will be loaded into this 
base scenario file. 

• Within the Excel RSG tool, the user specifies the name of the squad he 
wishes to create and assigns the force type, the waypoints to be used, and 
the communications links to be generated. 

• With the click of a button, with user-provided inputs, the scenario will be 
generated. 

Appendix A consists of detailed instructions on how to use the RSG tool. 

A diagrammatic flowchart, showing how the RSG tool is implemented, is shown 

in Figure 19. 


Executed in MANA Executed in EXCEL 



Figure 19. Architecture of Rapid Scenario Generation Tool 
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4. Upper Limits on the Processing Capability of MANA 

As this tool is able to rapidly populate forces for the scenario, resulting in 
thousands of agents and communications links created within minutes, the upper limit on 
the number agents that MANA can process was reached, as it throws an out-of-memory 
exception. The developers of MANA (Galligan, 2008) provided an upper limit on the 
program’s processing capabilities with the following parameters: 

CONSTANTS: 

Num_MultiRuns = 100000; 

Num_Steps = 100000; 

Num_States = 59; 

Num_Wpn =6; // Number of weapons allowed 

Num_Sens =6; // Number of sensors allowed 

Num_MOE = 5; 

Num_Wgts = 32; 

Num_Ranges = 77; 

Num_Tags = 326; 

Num_Start_Tags = 14; 

Num_End_Tags = 15; 

Num_TabVal = 30; // Number of Sensor/Sskp/Target AgtClass/Non 

Targets accepted in table 

Max_WPs = 100; 

Max_Homes = 100; 

Max_RecAgts = 999; 

Max_Integer = 2147483647; 

Num_BuiltinTerrains = 6; 

One of the key issues with managing huge models is the need to be prudent with 
the use of memory. One of the important factors was communications links and, in 
particular, the queue buffer size. This is the number of messages that can queue up when 
the capacity of the communications link is reached. By default, MANA sets the queue 
buffer size to infinity. In order to ensure that the thousands of communications links do 
not consume all the memory, there is a need to set this to a finite number. 

Other than the communications links’ characteristics, it is also important to design 
the communications topology to ensure that the memory is efficiently used. If there is a 
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need for multiple squads within a unit to communicate among each other, in a small 
scenario the many-many topology can be used, since memory is not likely to be an issue. 
However, in a large scenario with many communications links, a more preferred topology 
would be many-one-many. This involves the creation of a dummy relay agent that relays 
the messages. Figure 20 shows the difference between the two types of topology. We can 
see that for the many-many communications topology, the complexity of the 
communications grows in 0(n 2 ), in polynomial order two rate, whereas in the many-one- 
many topology, it grows in a linear rate of O(n). 



Figure 20. Communications Topology 
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III. EXPERIMENTAL DESIGN 


A. INTRODUCTION 

Simulation models are complex and can contain a huge number of input variables, 
of which a large number may have significant impact on the MOEs. In addition, many of 
these input variables are uncertain and their exact values cannot be easily quantified. 
Furthermore, the response surfaces may be nonlinear (Cioppa, 2002). Hence, it is 
important that the input variables are sampled across their entire space, while minimizing 
the amount of design points required to run the simulation experiment. This chapter 
discusses the variables that are studied, followed by an explanation of the designs used 
throughout the research. Finally, the processes of running the experiment are discussed. 

B. VARIABLES OF INTEREST 

In this thesis, the variables that are studied are capability focused. That is to say, 
only system and equipment characteristics (e.g., probabilities of hit, speed, armor, etc.) 
are studied. There are two main types of variables in this simulation: controllable and 
uncontrollable. Controllable factors are referred to as decision factors, those are things 
decision makers can influence and control. Uncontrollable factors are referred to as noise 
factors, and include such factors as environmental conditions, enemy capabilities, etc. 

In this thesis, the decision and noise factors are taken from two perspectives, one 
from the perspective of the defender and the other from the attacker. Hence, for the 
attacker, his decision factors include those of his own systems’ capabilities, and noise 
factors include those of the defender’s capabilities and environmental factors, such as 
population density. Common to both scenarios, the system capabilities factors listed in 
Table 3 are explored for both the attacker and defender. In addition, the environmental 
factors of cover and concealment are also explored. 

The key difference in the factors explored in the UO and RCO scenario is that in 
the UO scenario, the additional factor of civilians is explored. In addition, in the UO 
scenario, the sensor latency factor is not explored for the defender. 


29 



S/N 

Factor 

Remarks 

1 . 

Probability of Hit - Small Arms 

System 

Capabilities 

2. 

Probability of Hit - Artillery 

3. 

Probability of Hit - Antitank Weapons 

4. 

Artillery Shot Radius 

5. 

Artillery Rate of Fire 

6. 

Obstacle Clearance Rates 

7. 

Armor - Number of Hits to Kill 

8. 

Stealth 

9. 

Communications Latency 

10. 

Number of Artillery Guns 

11. 

Sensor Latency 

12. 

Speed 

13. 

Terrain - Cover 

Environment 

Factors 

14. 

Terrain - Concealment 


Table 3. Common Factors Explored for both Attacker and Defender 


1. Controllable F actors 

This section describes the factors that are being explored and their associated 

agents. 


a. Probabilities of Hit (Small Arms, Artillery, Antitank Weapons) 

The probabilities of individual weapon hits are varied to explore the 
effects of the accuracy of different classes of weapons. For small arms, the weapons that 
are being explored include those weapons that are being held by the individual soldier 
and machine guns. These weapons include the Ml6, AK47, Light Machine Gun (LMG), 
and General Purpose Machine Gun (GPMG). In terms of antitank weapons, these include 
the RPGs, Javelin, and 125mm tank guns. For artillery, it is just the artillery guns of the 
attackers and defenders. 

b. Artillery Shot Radius 

This factor is intended to help explore the effectiveness and importance of 
indirect fire weapons with small collateral damage. The damage radius is explored from a 
range of lm to 500m. 
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c. Artillery Rate of Fire 

This factor is intended to look into importance of rate of volumes of fire 
delivered onto a target, i.e., the frequency at which the target is being engaged. In MANA, 
this is modeled as a probability of engagement per time step. Hence, for a value of 50, 
this indicates that the chance that a target will be engaged, given that it is identified and 
classified, is 50%. For values over 100, such as 130, it means that one target will be 
engaged every time step with a probability of 100%, and a second target may be engaged 
with a probability of 30%. 

d. Obstacle Clearance Rates 

This factor explores the rate at which obstacles are being cleared. The 
values that are being assigned take on a meaning similar to that mentioned in the Artillery 
Rate of Fire. Every obstacle is assigned a certain number of hits to kill and assigning a 
clearance rate of 100% would mean that once an obstacle is detected, at every time step, 
it would be hit once, and once the obstacle’s number of hits is reached, it will become 
cleared. This factor is applied only to the engineer force, which, in the scenario, consists 
of only the bulldozers and minesweepers. 

e. Armor - Number of Hits to Kill 

This factor explores the role armor has in operations. It represents the 
number of hits an armored vehicle can withstand before it is killed. This factor is varied 
only for the infantry fighting vehicles and tanks. It is varied at a range from 1-10. 

f Stealth 

This factor describes the probability of being detected by another agent 
and is only applied to dismounted troops. 

g. Communications and Sensor Latency 

This factor explores the importance of timely communications and 
transmission of information on the battlefield. For this factor, the larger the latency, the 
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longer it takes for a particular message to be transmitted to the destination unit. In both 
scenarios, the latency is expressed in seconds; hence, for the experimental design; we are 
varying the latency to be between 0 (instantaneous) and 900 seconds (15 minutes). For 
the communications latency, it is applied to all communications links between troops. As 
for the sensor latency, it is only applied to sensors such as Unmanned Aerial Vehicles 
(UAVs) and counterbattery radars. 

h. Number of Artillery Guns 

This is the number of artillery guns that are being deployed on the 
battlefield and is varied to explore if there is a critical mass of guns that need to deployed 
before fire support can become effective in determining the battle’s outcome. 

i. Speed 

This is the speed at which vehicles travel on the battlefield and is applied 
to infantry fighting vehicles and tanks. 

2. Noncontrollable Factors 

a. Terrain - Cover 

This measures the ability of the terrain to provide protection from units 
firing on it. This is only applied to the buildings and urban area type of terrain for the UO 
scenario. For the river crossing scenario, it is applied to the buildings, light bushes, and 
urban terrain. 


b. Terrain - Concealment 

This measures the ability of the terrain to provide concealment from units 
firing on it so that they become less detectable. This is only applied to the buildings and 
also urban area type of terrain for the UO scenario. For the river crossing scenario, it is 
applied to the buildings, light bushes, and urban terrain. 
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c. 


Civilian Density 


This factor is explored only in the UO scenario and this factor setting 
determines the number of civilians in the AO. The purpose of this parameter is to study 
whether a densely or sparsely populated AO has an impact on the outcome of the battle. 

C. EXPERIMENTAL DESIGN AND ANALYSIS PROCESS 

Simulation modeling and analysis is done in an iterative manner to ensure that the 
model functions properly and that the output data is the result of sound assumptions and 
correct models. An initial exploratory design of a small number of factors is implemented 
to gain familiarity and understanding of the experimental design process and to identify 
potential bugs in the model. This is done only with the UO scenario, as the same model 
components are being reused in the river crossing scenario. Finally, the full experiment 
where the ranges of the input variables are refined to a tighter bound is run to obtain the 
final set of results for analysis. 

1. Nearly Orthogonal Latin Hypercube (NOLH) 

The NOLH was developed by Lieutenant Colonel Thomas Cioppa, United States 
Army, in 2002. It allows us to efficiently explore simulations with readily available 
experimental designs. It is a method that considerably improves the space-filling 
properties of the Latin hypercube, while inducing only small correlations among columns 
in the design matrix (Cioppa, 2002). The traditional factorial design, with only high and 
low settings, assumes linearity within the factors and is not able to explore nonlinearities, 
which are prevalent in simulation models. An NOLH generation tool in Excel, created by 
Professor Susan Sanchez at Naval Postgraduate School (NPS), is used to generate the 
NOLH designs for this thesis. The detailed NOLH designs used in this thesis are 
explained in Appendix B. 

2. Exploratory Design 

An exploratory design with only 11 factors (33 design points) was carried out 
using the UO scenario to gain familiarity on experimental design, as well as to gain some 
initial insights into the model. In this model, only a subset of factors is explored and some 
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factors are being aggregated (such as probability of hits). A total of nine replications are 
being carried out for this exploratory design. The factors that are being explored are listed 
in Table 4, with the scatter plot matrix shown in Figure 21. 


S/N 

FACTOR 

MIN VALUE 

MAX VALUE 

REMARKS 

1. 

Weapon Probability of Hit 

0.00 

1.00 

BLUE 

2. 

Communications Latency 

0 

900 sec 

3. 

Speed 

0 

72 km/h 

4. 

Armor (number of hits) 

1 

10 

5. 

Stealth 

0% 

100% 

6. 

Weapon Probability of Hit 

0.00 

1.00 

RED 

7. 

Communications Latency 

0 

900 sec 

8. 

Speed 

0 

72 km/h 

9. 

Armor (number of hits) 

1 

10 

10. 

Stealth 

0% 

100% 

11. 

Terrain - Cover 

0.00 

1.00 

Noise Factor 


Table 4. Summary of Factors for the Exploratory Design 
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BLUE Terrain- BLUEComms REDComms 

BLUEP(hit) BLUE-Armr Stealth Cover Latency BLUE Speed REDP(hit) RED-Armr RED Stealth Latency 


Figure 21. Exploratory Design Scatterplot Matrix 

3. Full Design 

a. Urban Operations (UO) 

Ten replications were made for the UO scenario; with 257 design points 
for 27 factors, a total of 2,570 runs of the scenario were made. Table 5 shows the 
summary of all the factors with their maximum and minimum ranges that are explored in 
the UO scenario. The scatter plot matrix for the full design of the UO model is shown in 
Figure 22. 
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S/N 

Factor 

Min 

Value 

Max 

Value 

Remarks 

1. 

Probability of Hit - Small Arms 

0.01 

1.00 

ATTACKER 

2. 

Probability of Hit - Artillery 

0.01 

1.00 

3. 

Probability of Hit - Antitank Weapons 

0.01 

1.00 

4. 

Artillery Shot Radius 

lm 

500m 

5. 

Artillery Rate of Fire 

1 

200 

6. 

Obstacle Clearance Rates 

1 

100 

7. 

Armor - Number of Hits to Kill 

1 

10 

8. 

Stealth 

1% 

100% 

9. 

Communications Latency 

0s 

900s 

10. 

Number of Artillery Guns 

1 

12 

11. 

Reserves Time of Activation 

0 

1800s 

12. 

Sensor Latency 

0s 

900s 

13. 

Speed 

1 

200 

14. 

Probability of Hit - Small Arms 

0.01 

1.00 

DEFENDER 

15. 

Probability of Hit - Artillery 

0.01 

1.00 

16. 

Probability of Hit - Antitank weapons 

0.01 

1.00 

17. 

Artillery Shot Radius 

lm 

500m 

18. 

Artillery Rate of Fire 

1 

200 

19. 

Stealth 

1% 

100% 

20. 

Armor 

1 

10 

21. 

Communications Latency 

0s 

900s 

22. 

Reserves Time of Activation 

0 

1800s 

23. 

Speed 

1 

200 

24. 

Number of Artillery Guns 

1 

6 

25. 

Terrain - Cover 

0.01 

1.00 

Environment 

26. 

Terrain - Concealment 

0.01 

1.00 

27. 

Civilian Density 

0 

600 


Table 5. Factors Used in the Experimental Design for Urban Operations Scenario 
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Figure 22. Scatterplot Matrix for the UO Model 


b. River Crossing Operation (RCO) 

Three replications were made for the RCO scenario, with 257 design 
points for 27 factors, for a total of 771 runs of the scenario. The extremely small number 
of replications chosen was a result of the large size of the model taking an extremely long 
time to complete one single run. From the initial runs, it is estimated that it takes 
approximately 24 hours to complete a single run of the scenario. Since the scenario is the 
equivalent of a 12-hour battle in actual operational situations, the ratio of CPU run time 
to real-world time is 0.5:1. 
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Hence, for a single replication, it requires 6,168 hours of CPU run time to 
complete, taking one week to finish on the DSO high-performance computing cluster. 

As it is desired to have a higher resolution on the factors influencing the 
battle outcome in an RCO, the larger design (257 design points) was chosen instead of a 
smaller one (33 design points). The choice was taken with the understanding that only 
very strong effects will be identified from the output analysis and trade-offs will have to 
be made on statistical power. Further studies and more runs, based upon the larger 
design, can be made in subsequent studies after the completion of this thesis. Table 6 
shows the summary of all the factors and their minimum and maximum values. 


Table 6. Factors used in the experimental design for the RCO Scenario 


S/N 

Factor 

Min 

Value 

Max 

Value 

Remarks 

1. 

Probability of Hit - Small Arms 

0.01 

1.00 

ATTACKER 

2. 

Probability of Hit - Artillery 

0.01 

1.00 

3. 

Probability of Hit - Antitank Weapons 

0.01 

1.00 

4. 

Artillery Shot Radius 

lm 

500m 

5. 

Artillery Rate of Fire 

1 

200 

6. 

Obstacle Clearance Rates 

1 

100 

7. 

Armor - Number of Hits to Kill 

1 

10 

8. 

Stealth 

1% 

100% 

9. 

Communications Latency 

0s 

900s 

10. 

Number of Artillery Guns 

1 

12 

11. 

Sensor Latency 

0s 

900s 

12. 

Speed 

i 

200 

13. 

Probability of Hit - Small Arms 

0.01 

1.00 

DEFENDER 

14. 

Probability of Hit - Artillery 

0.01 

1.00 

15. 

Probability of Hit - Antitank weapons 

0.01 

1.00 

16. 

Artillery Shot Radius 

lm 

500m 

17. 

Artillery Rate of Fire 

1 

200 

18. 

Stealth 

1% 

100% 

19. 

Armor 

1 

10 

20. 

Communications Latency 

0s 

900s 

21. 

Reserves Time of Activation 

0 

1800s 

22. 

Speed 

1 

200 

23. 

Number of Artillery Guns 

1 

6 

24. 

Sensor Latency 

0s 

900s 

25. 

Terrain - Cover 

0.01 

1.00 

Environment 

26. 

Terrain - Concealment 

0.01 

1.00 
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D. AUTOMATED RED TEAMING (ART) 

Red Teaming is a commonly used technique in military operations to uncover 
system vulnerabilities and discover weaknesses in military operational concepts. It is a 
manual process that involves subject matter experts coming together to analyze a given 
scenario and coming up with situations that are most threatening to BLUE forces. This is 
often tedious and is limited to the knowledge of the subject matter experts. ART is a 
framework that was developed by the DSO, Singapore. It employs evolutionary 
algorithms to discover system vulnerabilities and weaknesses in military operational 
concepts by automatically using parallel computing techniques. The main purpose of it is 
to reduce surprise, improve and ensure the robustness of Blue’s operational concepts 
(Choo, Chua, & Tay, 2007), and complement the strengths of the traditional Manual Red 
Teaming technique. 

1. Explore Tactics, Techniques, and Procedures (TTPs) 

The purpose of using ART in this scenario is to explore the TTPs of defending 
forces in urban areas. Typically, in any defense layout, there will be two types of forces: 
static and dynamic. Static forces are the forces that are deployed to hold and cover 
grounds of tactical importance (GTI). These are the forces that are the positional elements 
that provide the frame of a defense layout. Dynamic forces are those that are held in 
reserve and not committed. Therefore, they can be employed to carry out a wide variety 
of tasks and are most often used to seize initiative from the attacking force when the 
situation presents itself. The typical defense layout, shown in Figure 23 is extracted from 
FM3-90. 
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Figure 23. Defense Layout (From Headquarters, Department of the Army, 2001) 


The focus of using ART is to explore positions where RED static forces will be 
deployed and reinforcement routes that will be the most threatening to the attacking 
BLUE forces. 


2. ART Experimental Design 

Given the BLUE plan of advancing along two axes and the location of the RED 
static forces, the reinforcement routes of the RED reserve forces is being explored here. 
Table 7 shows the factors that are being explored. 

Table 7. ART Experimental Design Factors 


S/N 

FACTOR 

MIN 

VALUE 

MAX 

VALUE 

REMARKS 

1 . 

Static Force X coordinates 

0.00 

1.00 

This is applied for each of the 

18 agents in the RPG platoon, 
which forms the static force. 
This is to say there will be 36 
factors (18 X-factors, 18 Y- 
factors). 

2. 

Static Force Y coordinates 

0.00 

1.00 

3. 

Reserve Force i th Route X- 
coordinate ( where i=l,...,12) 

0.00 

1.00 

For the reinforcement routes, 
there are 12 waypoints 
altogether. This makes up a 
total of 24 factors (12 X- 
factors, 12 Y-factors). Within 
each of these factors, the 
waypoints of the reserve forces 
are lockstepped together. 

4. 

Reserve Force i th Route Y- 
coordinate ( where i=l,...,12) 

0.00 

1.00 
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E. 


RUNNING THE EXPERIMENT 


MANA uses XML for storage of information relating to the model scenario. With 
the information stored in XML format, MANA can be readily data-farmable on 
supercomputer clusters. Running the experiment is a two-step process, which first 
involves setting a special study file that specifies which entities within the model have 
specific characteristics (e.g., probability of hit) that are varied. Also, similar to MANA, 
the ART study file is also specified in XML format to facilitate data farming on clusters. 
Secondly, with the respective study files, they are sent to the cluster to carry out runs. 

1. Creating the Study File 

a. NOLH Experimental Design 

The tool XStudy, which was created by Steve Upton, Research Associate 
within the NPS Simulation Experiments and Efficient Designs (SEED) centre, is a very 
useful tool in assisting users to create their study files. It can be downloaded at 
http://harvest.ups.edu/ under software downloads, data farming tools. It is complete with 
a user guide and is a user-friendly tool using drag and drop functionality. At the end of 
the process, XStudy automatically generates a zip file that contains all the files required 
for the run and can be sent to the cluster manager for runs. Figure 24 shows a screenshot 
of the XStudy tool. 



Figure 24. Screenshot of XStudy 1.0 
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This tool is good for small models, where the parameters to be studied can 
be selected in a couple of mouse clicks. In large models, involving hundreds of squads 
whose characteristics must be varied simultaneously (also called lockstepping), additional 
tools are required to augment XStudy. 

As part of the RSG tool in an Excel spreadsheet, a separate design of 
experiments functionality was also added so that study files can also be rapidly created. It 
involves first using XStudy to create a sample study file. With that sample file, all the 
parameters inside are manually deleted through the use of notepad or any text editor. 
Subsequently, the factors, squads, and paths to be varied are specified in the spreadsheet, 
and with a click the study file is generated. 

b. ART Experimental Design 

The ART tool developed by DSO has a graphical user interface that allows 
users to specify the factors to be optimized and also provides the architecture for runs to 
be carried out on a single computer. Figure 25 shows a screenshot of the ART graphical 
user interface. 



Figure 25. Screenshot of the Automated Red Teaming Tool 
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2. Running on Cluster of High-Performance Computers 

The study file, MANA scenario file, NOLH file, and terrain maps are sent to the 
cluster manager for runs. In this thesis, two computing clusters were employed in 
assisting the generation of results. The first one is the NPS SEED cluster managed by 
Steve Upton and Mary McDonald, and the other is the computing cluster managed by 
Dave Ang and Chua Ching Lian, DSO, Singapore. The cluster of high-performance 
computers conducted the simulation in both the exploratory and full designs. The UO full 
design took about one week to generate 10 replications of 257 design points, which is 
2,570 runs of the scenario. The RCO full designs took about one week to generate just 
one replication of 257 design points. 
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IV. DATA ANALYSIS 


This chapter describes the process of collating and the processing of the data 
generated from the runs for analysis. Insights that are gained from the analysis are also 
discussed. 

A. DATA COLLECTION AND PROCESSING 

The output file from MANA comes in the format of comma separated values 
(CSV) files. The summary output files consist of information on the casualties for each of 
the RED and BLUE forces, as well as the state (dead or alive) of individual squads. Each 
design point comes with a summary output file that shows the output of the respective 
replications. In order to process the data from the large number of output files (257), a 
program was written using VBA to collate the output into one summary output file that 
could be read into statistical packages for analysis. This summary output file contains 
information on the design points, as well as the resulting output of the various 
replications of the individual design points. Analysis was done using JMP 7.0, a 
statistical package. 

B. CLUSTER AND OUTLIER ANALYSIS FOR DATA MINING (COADM) 

The statistical analysis is supplemented with a visualization tool developed by the 
DSO National Laboratories in Singapore. The purpose of COADM is to provide an 
additional dimension to analysis of the output data. This analysis allows the analyst a 
quick overview of the “good” and “bad” clusters within the data and identifies the 
parameters that are associated with the respective clusters. In addition, COADM 
identifies the outliers in each of the clusters and attempts to discover “surprises” (Choo, 
Ng, & Chua, 2008) 

C. INSIGHTS INTO RESEARCH QUESTIONS 

In Chapter I, there are two main questions that need to be answered. These two 
questions are being addressed through the analysis of the output data, with insights that 
are gained. To recap, the two research questions are: 
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• How effective are communications, sensors, weapons, and platforms with 
regard to enabling the combat system of systems to complete its mission 
under the two types of scenarios? 

• How do the characteristics of the various terrain (such as urban, rivers) 
impact the performance of the combat forces? 

The questions will be addressed from the perspective of both the attacker and the 
defender. Several tools are employed to assist in answering these two questions. These 
include the use of regression analysis, as well as regression trees. Regression tress 
provide a simple and intuitive summary to models containing a large number of variables. 

1. Urban Operations (UO) - Effects of System Capabilities 

a. Distribution of Red and Blue Casualties, Force Exchange Ratio 

Figure 26 shows the distribution of RED and BLUE casualties and the 
Force Exchange Ratios. 



Figure 26. Distribution of Red and Blue Casualties, and Force Exchange Ratio 
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From Figure 26, we see that overall, across a spectrum of system 
capabilities (both red and blue forces) and terrain conditions (civilians, cover and 
concealment), the attacker (Blue) suffers more casualties than the defender (Red). The 
average force exchange ratio is around 1.5 Blue forces for every Red force attrited (1.469 
from Figure 26), or more simply, for every two Red units attrited, three Blue forces are 
attrited. We can see that, on average, the urban battlefield is one that is disadvantageous 
to the attacker. 

In addition, we see that the variability in casualties suffered by the Blue 
forces is also 70% higher (standard deviation of 56 for Blue versus 33 for Red) than that 
of the Red forces. A plot of red casualties against blue casualties is shown in Figure 27. 
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Figure 27. UO Scenario - Plot of Red Casualties against Blue Casualties 


From Figure 27 we can see that for mission success, which is defined as 
inflicting at least 67% Red casualties and losing less than 67% of Blue forces, over a 
wide spectrum of capabilities and terrain conditions, only about 92 design points out of 
the 252 are mission successes for the attacker. This translates to an approximately 26% 
chance of mission success in a UO scenario. 
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b. Factors Influencing Extent of Red Casualties 

Figure 28 shows the regression tree analysis on factors influencing the 
defender casualties. 



Figure 28. Regression Tree Analysis on Factors Influencing the Defender Casualties 


From Figure 28, we can see that cover provided by the terrain has the most 
impact in terms of influencing the extent of red casualties. Without sufficient cover from 
the surrounding terrain, in this case the buildings within the urban terrain, the next factor 
that then becomes important is Blue’s armor protection capability. This is measured by 
the number of hits the armored vehicle can sustain before being attrited. From the 
regression tree, this number is three. 

This is a particularly interesting observation, as we can see that in an 
urban combat scenario, it is important to have good survivability before you are able to 
successfully inflict casualties on the enemy. The lethality of weapons is secondary to that 
of survivability. From observing a couple of the simulation runs of the UO scenario, it 
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can be seen that it is important to be able to provide sufficient armor protection to the 
attacking forces in the advance to the objective to fight the defending red forces. 

For defending forces, it means that better armor-defeating capabilities 
(better antitank weapons) would be the key to reducing their own casualties. 

In addition, the exploitation of terrain in providing cover and concealment 
will contribute to lower casualties as well. 


c. Factors Influencing Extent of Blue Casualties 

Figure 29 shows the regression tree analysis on factors influencing the 
attacker’s casualties. 



Figure 29. Regression Tree Analysis on Factors Influencing the Attacker Casualties 


As for the casualties suffered by the attacking Blue forces, the first critical 
factor is, in fact, sensor latency. The sensors that this factor specifically refers to are the 
UAV, as well as the counterbattery radar that the battery supporting the attacking 
mechanized infantry battalion is equipped with. From Figure 29, we can see that a latency 
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in delivering information from the sensor to the shooter of more than 1.5 minutes 
(98 seconds) almost doubles the mean casualties (from around 90 to 170) suffered by the 
Blue forces. This demonstrates the importance of not just accurate information, but also 
information that is timely. 

A rerun of a couple of simulation runs indicates that when information is 
delivered with a long delay, it often becomes obsolete and subsequent actions, such as 
calling for artillery fire, will result in attacks on a nonexistent target. This is especially 
critical when targeting mobile reserves, where the target window of opportunity opens 
and closes very rapidly. In observing the simulation runs, it is seen that if the targeting 
effort on the defenders’ reserves is not executed well, it could result in the attacking 
forces having to deal with the reserves that leaked from the targeting effort, thus 
increasing casualties in the ensuing combat at the frontal positions. This is especially so 
for armored forces, where they move very fast. 

From the output, we also note that if the latency in the attacker’s 
sensor-shooter chain is more than approximately 1-2 minutes, the attacker must move 
slower to avoid casualties. The number of casualties sustained by the attacker is halved 
(from 183 - 95) if the attacker moves slower. The results indicate that the movement 
speed should be slowed to around 1 m/s (13 grids in 10 seconds) or about 4km/hr, which 
is approximately the marching speed of infantry troops. A review of the simulation runs 
indicate that given a higher latency, there is a need to allow time for the artillery fires to 
soften the defender positions before the troops move in to secure the objective. 

In addition, with higher sensor-shooter latency, the defender’s mobile 
reserves could not be easily targeted en route to the forward positions. A slower speed 
would trigger the defender’s reserves to the forward positions first and without the 
attacker engaging in decisive combat with the deployed reserves. The defender’s 
deployed reserves would then be exposed to artillery fires from the attacker. This will 
make it easier for the attacker to capture the objectives once they arrive, albeit at a slower 
speed. 
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From the regression tree analysis, it appears that for the defending Red 
forces, the most important factor to increasing Blue casualties is responsive 
communications. What this means for the attacking Blue forces is that by jamming 
communications of the Red forces prior to attacking, it is expected that a lower casualty 
rate can be achieved. 


d. Factors Influencing Extent of Force Exchange Ratio 

Figure 26 shows the distribution of RED and BLUE casualties and the 
Force Exchange Ratios. 



Figure 30. Regression Tree Analysis on Factors Influencing the Force Exchange 

Ratio 


We can see from Figure 30 that given terrain is neutral, that is it does not 
feature itself within the regression tree analysis. The factors that were analyzed in Blue 
and Red casualties also are dominant factors influencing the force exchange ratio as well. 
In addition, we can see that weapon lethality of antitank weapons is the additional factor 
that comes into effect for force exchange ratios. 
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The regression tree analysis revealed that when the armor of the attacker is 
able to sustain at least two or more antitank hits, the force exchange ratio turns 
dramatically in favour of the attacker, from 3.0 to 1.2, a more than 50% improvement. 
Incidentally, the result also reveals that when armor protection is insufficient for the 
attacker (less than two hits), the traditional 3:1 ratio of attacker-defender applies in this 
UO scenario. 


e. Factors That Can Influence Variability in Battle Outcome 

Figure 31 shows the regression tree analysis on factors influencing the 
variability in force exchange ratio. 



Figure 31. Regression Tree Analysis on Factors Influencing the Variability in the 

Force Exchange Ratio 


We want to identify capabilities that will minimize the variability in battle 
outcomes. This is done by considering the standard deviations of the force exchange ratio 
against all the input parameters. From Figure 31, we can see that the two main factors are 

communications responsiveness of the defending Red forces and the armor of the 
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attacking Blue forces. These are also the two factors that have effect on the mean of the 
force exchange ratio as well. Hence, the robust strategy for the attacking Blue force 
would be to focus on better armor protection and employing jamming techniques to 
degrade the communications of the defending Red force. Conversely, for the defending 
Red force, it would be to explore the use of better anti-armor weapons, deploy effective 
communications links, and ensure that the communications links are not susceptible to 
jamming by the attacker. 

/. Cluster and Outlier Analysis 

Figure 32 shows an overview of the factor settings in the Design of 
Experiments, as well as the distribution of the MOE output over all the design points. 



Figure 32. Overview of Correlation Plots for Urban Operations 
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The MOEs that are studied in this COADM analysis are the Mean Force 
Exchange Ratio, and Mean Red and Blue Casualties. The rectangle that goes from Blue 
to Red on the right side of the colored patterns represents the factor settings. Blue 
represents low and Red represents high. The colored patterns represent how the high and 
low settings are distributed. The various factors can then be read by comparing against 
other factor settings or by comparing with the MOE output. For instance, for the three 
MOEs, we can see that with a high mean Blue Casualty, low mean Red Casualty, the 
Mean Force Exchange Ratio will be high. 

From the correlation plots with different colored patterns, we can see that 
there is little correlation between the input factors. This is expected, since the NOLH 
designs ensure almost orthogonality among the input factors. Figure 33 shows the factors 
that appear to have high correlation with the MOEs, i.e., the colored patterns of the 
factors vary very closely with changes in the MOEs. 



Figure 33. High Correlation between Factors and MOE for Urban Operations 
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The factors that are identified as important are: 

• Blue’s antitank weapons probability of hit 

• Blue’s armor protection 

• Blue’s sensors’ latency in information transmission 

• Red’s antitank weapons probability of hit 

• Red’s communications latency 

These are factors that were also identified in the regression tree analysis 
and are consistent with that analysis. Figure 34 shows the outliers that are identified in 
the respective clusters in the output data. 



Figure 34. Outlier Identification in Urban Operations Model 


Taking into consideration the MOEs and also the input factors, CO ADM 
generated a total of 10 clusters. Within these 10 clusters, 10 outliers were identified and 
are found in clusters 2,4,7,8,9,10. In this outlier analysis, we will classify clusters where 
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the mean force exchange ratio is less than 1.0 as “clusters.” This means that it takes less 
blue casualties to exchange for red casualties. Table 8 shows the mean force exchange 
ratio of the various clusters. 


Cluster 

S/N 

Mean Force 
Exchange 
Ratio 

Mean Red 
Casualties 

Mean Blue 
Casualties 

Outliers 

1 

0.832 

161.400 

125.000 

Nil 

2 

1.012 

131.500 

121.300 

231 

3 

1.531 

105.300 

174.900 

Nil 

4 

2.120 

95.700 

197.200 

167 

5 

1.405 

143.300 

200.700 

Nil 

6 

1.286 

140.700 

178.700 

Nil 

7 

1.771 

117.400 

218.700 

165, 204 

8 

0.728 

133.900 

90.200 

222 

9 

1.120 

158.400 

175.400 

16, 8, 108 

10 

1.179 

101.400 

120.900 

2,4 


Table 8. CO ADM Individual Cluster MOE 


We can see that in the table, there are only two “good” clusters, which 
corresponds to clusters 1 and 8. This also agrees with the analysis that in an urban combat 
scenario, the exchange ratio favors the defender and that the attacker will, more often 
than not, need to devote more forces for an urban fight. 

The next analysis is on looking for “good” outliers in “bad” clusters and 
also “bad” outliers in “good” clusters. In this analysis, a “good” is defined as having a 
force exchange ratio that is less than one. This can be interpreted as having an exchange 
ratio that is in favor of the attacker. This analysis seeks to uncover any possible surprises 
that may come out as a result of unexpected combinations in the input variables. The 
detailed output data is shown in Appendix C. From analysis of the individual outlier data, 
we see that there is only one “bad” outlier in the “good” clusters. It is identified as point 
222, with a force exchange ratio of 1.87 in a cluster where the average is 0.728, with a 
standard deviation of 0.248. A review of the settings show that in point 222, the antitank 
weapon probability of hit for the attacker is set at 0.0001, which is effectively almost zero. 
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We see that the since the antitank weapon probability of hit is one of the key factors in 
influencing the force exchange ratio, it is likely that this setting caused the outlier. 


There are three good outliers in two “bad” clusters, they are 165 and 204 
in cluster 7 and 231 in cluster 2. For points 165 and 204 (with force exchange ratio of 
0.35 and 0.43 with a cluster average of 1.286), it was found that in these two points, the 
attacker moves at very low speed (the setting is equivalent to approximately half the 
walking speed of lm/s) and the sensor latency is approximately 5 minutes. This agrees 
with the previous analyses that when there is a high sensor latency of more than 1-2 
minutes, the attacker should advance at speeds lower than walking speed. This will result 
in a reduction of the attacker’s casualties by 50%. In the case of point 231 in cluster 2 (at 
0.93 with a cluster average of 1.012), it is likely that the outlier was caused by a Blue 
stealth setting of 100%. 

g. Red Teaming (Manual and Automated) 

In this section, the results from red teaming (defender) are discussed. A 
manual red teaming was carried out in International Data Farming Workshop 17, held in 
Garmisch, Germany. The focus of the manual red teaming effort during that study in the 
workshop was to explore the TTPs of the defender in an urban terrain with the aim to 
minimize own casualties and maximize the attacker’s casualties. Specifically, the aim 
was to gain insights into how the defender should deploy his static forces and also how he 
should deploy his reserves. The results show that by uniformly distributing his static 
forces along the main axes and deploying his reserves from the flanks of the attacker, the 
defender can tilt the force exchange ratio to his advantage. This strategy is robust as well, 
since the standard deviation of the force exchange ratio is also the smallest. The detailed 
report for the workshop is as attached in Appendix D. 

ART employing the tool developed by DSO National Laboratories, 
Singapore was also applied on the UO scenario. However, the runs crashed prematurely 
as it was not able to handle a large scenario. DSO National Laboratories has taken steps 
to improve ART for use on larger scenarios. 
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2. River Crossing Operations (RCO) - Effects of System Capabilities 

As a result of computing resource constraints, only a single replicate is obtained 
for the output for the river crossing scenario. These findings should be considered 
exploratory, and follow-on studies can continue with more runs to produce findings that 
are statistically stronger. 

a. Distribution of Red and Blue Casualties, Force Exchange Ratio 

Figure 35 shows the distribution of RED and BLUE casualties and the 
Force Exchange Ratio for the river crossing scenario. 




Mean 0.9589101 

StdDev 1.0202863 

Std Err Mean 0.066415 
Upper 95% Mean 1.089755 
Lower 95% Mean 0.8280653 
N 236 


Mean 204.41102 

StdDev 85.083817 

Std Err Mean 5.5384847 

Upper 95% Mean 215.32244 
Lower 95% Mean 193.49959 
N 236 


Mean 232.64407 

StdDev 36.571126 

Std Err Mean 2.3805775 

Upper 95% Mean 237.33407 
Lower 95% Mean 227.95407 
N 236 


Figure 35. Distribution of RED and BLUE Casualties, and Force Exchange Ratio for 

RCO 
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From the distribution plot, we see that in general for an RCO, the mean 
exchange ratio is around 0.95 (about one attacker to every one defender casualty). Figure 
36 shows the plot of RED against BLUE casualties. 
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Figure 36. RCO - Plot of Red Casualties against BLUE Casualties 


From Figure 36, the green box shows the region where there was a 
mission success for the attacker. A count reveals that 123 of the 236 runs are mission 
success for the attacker. This implies that over a wide range of capability settings and 
terrain conditions, the attacker has a 52% chance of success in an RCO. 

When this is compared to the UO scenario, we see that the attacker has 
twice as many chances for success in an RCO than in UO. 

b. Factors Influencing Extent of Red Casualties 

Figure 37 shows the regression tree analysis for RED casualties in the 
river crossing scenario. 
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Figure 37. RCO Regression Tree Analysis for RED Casualties 


From Figure 37, we can see that terrain concealment is the first most 
important aspect in determining red casualties. By operating in terrain that has excellent 
concealment, RED is able to reduce his casualties by more than 30%. Terrain with 
excellent concealment allows the defender to stay undetected by the attacker’s sensors 
(consisting of both UAVs and artillery counterbattery radar). This reduces casualties, as 
the defender’s forces cannot be easily targeted. The next split corresponds to BLUE’s 
sensor latency, confirming that the attacker’s sensors are effective when the terrain 
concealment is low. What this means for the defender is that in a densely vegetated 
terrain, he should exploit the overhead foliage to reduce his vulnerability. For the attacker, 
the implication is that having sensors that can penetrate foliage will greatly enhance his 
targeting effectiveness and eliminate the defender’s advantage within the terrain. 
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In the case where both BLUE’s sensor and RED communications latency 
is more than 1.5-2 minutes, survivability of both the RED and BLUE forces becomes an 
important factor in determining the level of red casualties. This is obvious from the 
subsequent splits of BLUE armor—number of hits and terrain cover. We can see from 
Figure 37 that the number of antitank hits the attacker’s armor can sustain is an important 
factor in determining the defender’s casualties. The threshold of three hits that was 
identified in the RCO scenario is similar to that found in the UO scenario. 

c. Factors Influencing Extent of Blue Casualties 

Figure 38 shows the regression tree analysis for BLUE casualties in the 
river crossing scenario. 



Figure 38. RCO Regression Tree Analysis for Blue Casualties 
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From Figure 38, we can see that with the attacker having a sensor latency 
of less than 1.5 minutes will result in a casualty rate that is lower by more than 67% 
(from 222 to 69). The importance of the sensor is two-fold, one for the targeting of the 
Red reserves and secondly for the artillery battle (counterbattery fires) in the scenario. A 
successful counterbattery fire will reduce the exposure of the attacker to artillery fires as 
he crosses the river and in subsequent advance to the objectives. Effective targeting of the 
defender’s reserves will also deny the defender the opportunity to mass combat power at 
the crossing sites to defeat the elements of the attacker’s forces that have crossed the river. 

From the regression tree, we can see that if the targeting effort is not 
effectively carried out, where a significant portion of the reserves are able to mass at the 
forward positions, it is the number of hits the defender’s armor is able to sustain that will 
determine the attacker’s casualties. In cases where the targeting effort is not effective, 
closer-range weapons, such as antitank weapons, need to be more effective. This is 
evident from the split in the regression tree indicating the importance of a higher 
probability of hit for the antitank weapons of the attacker to reduce own casualties. 

Another important observation to note from Figure 38 is that when sensor 
and communications latency is high (>l-2 min) the attacker should advance at a rate that 
is almost equivalent to infantry marching speed. This is a finding that agrees with that 
concluded in the UO scenario. 

d. Factors Influencing Force Exchange Ratio 

Figure 39 shows the regression tree analysis on the force exchange ratio in 
the river crossing scenario. 
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Figure 39. RCO Regression Tree Analysis for Force Exchange Ratio 


From Figure 39, we can see that the key determining factor for the force 
exchange ratio is the attacker’s sensor latency. If the latency is less than 1.5-2 minutes, 
the mean exchange ratio is about three RED casualties for every BLUE casualty, which is 
very favorable for the attacker. If the latency is higher than that, the mean exchange ratio 
goes up to one red casualty for every BLUE casualty. 

From the regression tree, we can also deduce that when targeting is not 

effective, the attacker will need effective antitank weapons that will be able to take out 

the defender’s reserves that manage to mass at the forward positions. It is at this point 

where the number of hits the defender’s armor is able to sustain makes a difference to the 

force exchange ratio. From this observation, we can conclude that where possible, the 

defender should deploy reserves from armored vehicles (able to sustain at least three 

antiarmor hits) instead of soft-skin vehicles. 
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e. Cluster and Outlier Analysis 

Similar to the UO scenario, a cluster and outlier analysis is carried out as 
well. The detailed output data is shown in Appendix C. The MOEs are shown in the 
circled box in Figure 40. 
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Figure 40. Overview of Correlation Plots for River Crossing Operations 


From the correlation plots with different colored patterns, we can see that 
there is little correlation between the input factors. Again, this is expected as a feature 
due to the NOLH experimental designs. Figure 41 shows the factors that have high 
correlation with MOEs. 
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Figure 41. High Correlation between Factors and MOE for River Crossing 

Operations 

The input factors that have relatively higher correlation with the measures 
of effectiveness are listed as follows: 

• Attacker’s sensor latency 

• Attacker’s antitank weapons probability of hit 

• Number of hits the attacker’s armor can sustain 

• Defender’s communications latency 

• Terrain concealment 

This agrees with the findings from the regression tree analysis that was 
done prior to the cluster and outlier analysis. 

Figure 42 shows the identification of outliers in the output data for the 
river crossing scenario. 
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“Bad” outliers 



Figure 42. Outlier Identification in River Crossing Operations Model 


From the output from the outlier analysis, clusters 2, 4, 5, 6, 8 are 
identified as “good” clusters. These are clusters having a force exchange ratio of less than 
1.0. In addition, the outliers that are identified within the output data are either in the case 
of “bad” outliers in “bad” clusters or “good” outliers in “good” clusters. To put it simply, 
in clusters with a mean force exchange ratio of lower than 1.0 (“good”), all the outliers 
have force exchange ratios of lower than that of the cluster mean. 

In addition, in clusters with a mean force exchange ratio of higher than 
one (“bad”), all the outliers have force exchange ratios that are higher than the cluster 
mean. There are no “surprises” that we need to explore in the output data. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. RESEARCH SUMMARY 

Every system and weapon will have to operate with other weapons or systems to 
fight as a single entity during combat. This research sets out to propose a 
capability-focused framework, using a synthesis of different computer simulation 
techniques to analyze a system of systems, in order to better understand how they 
perform under various stressful operational environments. Using the river crossing and 
UO as the background for the scenarios, this thesis generated detailed analysis on the 
influence of the various types of capabilities on battle outcomes. The simulation work 
from this thesis provided the framework and also tools for the future use of ABMs in 
analyzing large-scale combat systems. 

B. RESEARCH QUESTIONS 

The goal of this thesis was to provide insights into the following questions: 

• How effective are communications, sensors, weapons, and platforms with 
regard to enabling the combat system of systems to complete its mission 
under the two types of scenarios? 

• How do the characteristics of the various terrain (such as urban areas, 
rivers) impact the performance of the combat forces? 

This chapter briefly summarizes the insights and answers into these questions. 

1. Effectiveness of Communications, Sensors, Weapons, and Platforms 

From the analysis, it revealed that there are two recurring observations in both 
scenarios. In the case for the attacker, the armor protection should be capable of 
sustaining at least three antitank hits. This produces higher casualties for the defender. 
The second recurring observation is that for targeting of the reserves to be effective, the 
latency of the sensors (e.g., UAV and counterbattery radar) in transmitting information 
back to the shooter (artillery units) will have to be less than approximately 1.5 to 2 
minutes. 
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If the sensor-shooter latency is more than that threshold, then it becomes 
important for the closer-range antitank weapons to be effective to take out the 
defender’s reserves. 

For the defender, it is observed that in both scenarios, communications latency is 
the main factor that is within the defender’s means to reduce their own casualties. Similar 
to that of the attacker, the threshold that is identified for the communications latency is 
1.5 to 2 minutes, as well. Both scenarios show that the defender has to exploit the 
characteristics of the terrain to his advantage. In the UO scenario, the cover afforded by 
the terrain is identified as the main terrain characteristic important to the defender, 
suggesting that the cover provided by buildings in an urban area is an advantage that the 
defender must exploit. In RCO, concealment is identified as the key characteristic. This 
suggests that the defender, where possible, will need to exploit the terrain concealment to 
hide the locations of his reserves to avoid being targeted by the attacker’s artillery. 

2. Impact of Terrain on Performance 

From the data, we can see that in a UO, the chances of success for the attacker 
across a spectrum of capabilities and terrain conditions are at 26%. This is much lower 
when compared to that of the RCO, which stands at 52%. Mission success is defined as 
attriting more than two-thirds of the defender’s forces, while retaining at least one-third 
of the attacker’s force to ward off subsequent counterattacks by the defender. 

The force exchange ratios for both the scenarios also support the findings. In the 
urban scenario, the attacker-defender mean force exchange ratio is 1.5 (1.5 attacker 
casualty for every defender casualty). For the RCO, that stands at a lower mean of 1.0. 

C. ADDITIONAL INSIGHTS 

In addition to the addressing the research questions, this thesis produced further 
insight as well. This section briefly summarizes the additional insights that were gained. 

1. Attacker - Advance Only As Fast As Sensor-Shooter Cycle 

Both scenarios revealed that where the sensor-shooter chain is slow (high latency), 
the attacking forces should advance slowly. When there is high latency, the defender’s 
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reserves will not be well targeted, and a significant number will leak out and deploy at 
the frontal positions. This tactic allows the defender’s reserves to be drawn out and 
deployed, thereby exposing them to targeting at their final deployed positions. Once the 
positions are “softened” by targeting and fires, the attacker will be able to capture these 
positions with less resistance. 

2. Attacker - Communications Jamming 

In both scenarios, we see that communications latency is one of the few factors 
that the defender is in control of in determining battle outcome. We can see that jamming 
of the defender’s communications can be an effective method in blunting one of the few 
sources of strengths of the defender. 

3. Attacker - Sensors That Can Penetrate Foliage 

From the analysis, it revealed that in the river crossing scenario, the attacker’s 
sensors are only effective when the concealment afforded by the terrain is lower. This 
suggests that sensors that are able to overcome foliage can be an important enabler for an 
effective targeting capability. 

4. Defender - Employ Armored Reserves 

From the analysis, it revealed that the defender’s reserves have a decisive impact 
on the battle outcome. We can see that where targeting is ineffective, the attacker’s 
casualties always increase due to the need to capture a better defended position. With that 
as a background, we can conclude that if the survivability of the defender’s reserves can 
be increased by deploying them via armored carriers, it will take the attacker a lot more 
targeting effort and direct hits on the reserves to render them ineffective. 

5. Simulating Operations 

The advantage of computer simulation is the ability to cheaply and quickly 
simulate numerous types of large scale operations without placing any soldier at risk. In 
addition, computer simulation allows us to evaluate the effectiveness of new systems that 
are not yet developed and provide insights into how these new systems can be operated 
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alongside existing systems. TTPs of these new systems can be easily studied using 
computer simulation. This thesis simulated 2,570 UOs and 257 RCOs in different terrain 
conditions and capability settings. The analysis of these results reveals lessons learned for 
these operations that would be costly in terms of money, time, and casualties in real life. 

6. Limits of Agent-Based Model (ABM) - MANA 4.0 

Through the use of rapid scenario generation, which provided the avenue for 
exploring one of biggest models ever attempted in MANA, this thesis was able to push 
the limits of MANA. It is found that MANA would not be able to handle a model that is 
beyond 1,000-plus agents and 2,000-plus communications links. When translated to force 
sizes, this means approximately a brigade-plus-sized scenario is the maximum MANA 
can handle. Beyond this size, first it would take a very long time to complete a single run 
and second, the user runs the risk of MANA running out of memory. 

D. RECOMMENDATIONS 

The results of this thesis support the following recommendations: 

• It is recommended that for armor to be effective, it will need to be able to 
withstand or deflect at least three antitank hits. 

• For targeting to be effective, the sensor latency in transmitting information 
to the artillery units will have to be less than 1.5 to 2 minutes. 

• For sensor latency that is higher than 1.5 to 2 minutes, the attacker will 
have to advance at a much slower pace. 

• In a river crossing operation, sensors that are able to penetrate overhead 
concealment (such as foliage from the vegetation) are important enablers 
for good targeting capability. 

• The attacker can improve his chances of success by jamming the 
defender’s communications prior to and during the capture of the objective. 

• The defender should improve the communications responsiveness of his 
forces. This can be achieved via better communications equipment and/or 
better training, to ensure that battlefield information is delivered promptly. 

• The defender should exploit the terrain characteristics to his advantage. 
Specifically, for UO, this is cover. For RCO, it is concealment for 
his reserves. 
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• The defender’s reserves have a significant impact on the battle outcome. 
Improving the survivability of these reserves, such as by using armored 
carriers to transport them, can increase the defender’s chances of success. 

E. FURTHER RESEARCH 

The following items are identified as areas warranting further research in: 

• Continuing with more replications on both of the scenarios to obtain 
conclusions that can be statistically stronger. 

• Explore the effects of different force structures. This can be done by 
reducing or increasing the force size. 

• Human intangibles, such as aggressiveness and teamwork, can be further 
explored to understand their impact on battle outcome at the systems level. 

• Expanding the scenario size of the urban scenario to a larger size at the 
brigade level. 
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APPENDIX A. THE RAPID SCENARIO GENERATION (RSG) 

TOOL 

This chapter describes how the RSG tool that is developed can be used. This tool 
is not necessarily useful for all MANA users. For users looking at small models, it is 
better to do it manually, as it may involve more work using the RSG tool. It covers the 
necessary steps to generate a huge scenario. This step-by-step instruction contains four 
sections (A-D). 

A. CREATE MODEL COMPONENTS IN MANA 4.0 

One of the first steps would be to create all your individual model components in 
MANA first. This step is important as it also allows for a pseudo-testing of the individual 
components’ behavior, and allows the modeler to see if the particular is behaving in the 
intended manner. Four types of MANA model files need to be created, each containing 
waypoints, generic behavior squads, communications squads, and lastly, the base 
scenario file, respectively. 

The waypoints MANA file contains all the possible planned manners in which the 
agents can move within the scenario map. The generic squads MANA file contains all the 
types of squads that you have within your scenario (for instance, infantry, tanks, etc.). 
The communications squad MANA file contains all the different communications 
characteristics that will be assigned to different types of forces within your scenario. The 
base scenario file is derived from the MANA scenario file and can be created by just 
manually editing the XML file and deleting all squads from the file. The base scenario 
file is the file in which all the squads will be inserted into. 

Once the three different types of MANA files are defined, the next step is to 
export the three different types of squads (waypoints, generic squads and 
communications) to three different folders. Before the squad files are exported, there are 
a few requirements that should be met: 
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• Unique Communications Link - For a communications squad, there 
should only be one type of communication link that the user would like to 
use. Multiple links will result in a program error. 

• Unique Squad Name - Each of the squads should have a unique name. A 
nonunique name will result in the wrong squad being replicated. 

• No Communications Links in Noncommunications squad files - In 

generic squad files and waypoint files, there should not be any 
communications link s . 

• No Squads in Base Scenario File - As mentioned, this base scenario file 
can be created by just taking any one of the three files (waypoints, generic 
squads, communications) and deleting all the squads from it. 

From MANA, this could be exported from one of the drop-down menus using the 
“Save All Squads.” A screenshot of that window is as shown in Figure 43. 



Figure 43. Screenshot from MANA on Exporting Individual Squads 

B. RAPID SCENARIO GENERATION (RSG) EXCEL INTERFACE 

With the basic component model files (base scenario, waypoint, generic squads, 
and communications) done, the next step is to import all the component files into the tool 
for generation of the MANA scenario file. This section explains the graphical user 
interface (GUI) of the RSG tool and the steps in using it to generate a scenario. The user 
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must ensure that the macro security settings are set to low and all macros are enabled for 
the scenario. This step is required so that the YBA code in the Excel sheet can work. 

There are two main parts to the tool, the first is for the importing of data into the 
generator and the second is for the user to create his scenario. Figure 44 shows the 
user-interface for the importing of squad files. 



Figure 44. User-interface for importing of squad definition files 

Each time before importing the squad files for a new scenario, click on the “Clear 
All” button for each of the three categories of squads. With that, click on “Import” for 
each of the categories to get the files required. The user will be prompted to select the 
directory and the files required for import. Multiple files can be selected by clicking and 
dragging of the mouse cursor over the files required at the directory. Figures 45 and 46 
show the pop-up screens for importing of squad definition files. 
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Figure 45. 


User-interface for importing of squad definition files (pop-up screen) 


|Q Microsoft Excef non-commerctal use - RSG (UrbanOps) vS.Ocaa 25 Oct Ofi ({derrmirrt ration for Thefts) [CompabtHlrty ModeJ 



Figure 46. 


User-interface for importing of squad definition files (pop-up screen) 
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Once the files are imported, the interface will look like Figure 46. The next step is 
the design of the scenario. Figure 47 shows the user interface for the design of the 
scenario. 


|Q Microsoft Ejecel non-commercial use - RSG (UibanOps) vSjO caa 25 CM OS ((demo nitration for Thesis) [Compatibility Monde] 
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Figure 47. User Interface for Design of Scenario 
The entries for the various columns shown in Figure 47 will need to be filled in. 


The columns’ functions and input requirements are as specified in Table 9. 


S/N 

Column Name 

Function 

Remarks 

1 . 

Squad Index 

Squad ordering numbering that 
is referenced within MANA. 

Ensure that all the numbers are 
in ascending order. Any 
repeated number will cause an 
error 

2. 

Squad Name 

Name of the particular squad 
that you are creating 

Ensure that there is no repeated 
name within the entire column. 

3. 

Parent Squad 

Specifies which squad is its 
parent. 

The specification is referenced 
by the squad index. 0 if there is 
no parent. 

4. 

Waypoint squad 

Specifies the waypoints of the 

There is a drop-down menu 
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S/N 


Column Name 


Function 


Remarks 


import 


squad that will be created. 


within each cell in this column. 
Select it from the drop-down 


VAYPOINT SQD 
IMPORT 


GENERIC SQD 
IMPORT 


Blue_A_PL1.sqd BLUEJFV.sqd 

Blue_A_PL1.sqd BLUE.Al.sqd 

Blue_A_PL1.sqd BLUE_IFV.sqd 

Blue A PLI.sqd BLUE_Al.sqd 


Sue. A_PL1.sqd 1 t UEJFV.sqd 
*|llE_Al.sqd 
UE_IFV.sqd 
UE_Al.sqd 
UE_IFV.sqd 
„ UE_Al.sqd 


_ft_PL2.r^ 

.B.PLImJ 

.B_PL 

>_B_PL3.m4 
BLU^C^ICauA^rj 
BLUE E< 


menu. 


Blu^A_PL2.sqd BLUEJFV.sqd 

Blue_A_PL2.sqd BLUE. Al.sqd 


Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 

Red.Comms 


Generic Squad 
Import 


Specifies which type of squad it 
is. Similar to the waypoint 
import, it can be selected from a 
drop down menu within each 
cell. 


GENERIC SQD 
IMPORT 


BLUE IFY.s< 


3±- 


ELULRw.Tri^trj-iJ 

ELULT-r.G<|J 

E: L U E_T r. l-:_ Inr.lyj J 

BLUE_UflVjqJ 


BLUE"iF'v.sqd 


6 . 


Communications 
Squad Import 


Specifies which type of 
communications link is being 
employed by this squad. 
Similarly, select the type from 
the drop-down menu. 


COMMS SQD IMPORT 


Red Comms Force Met.sqd 


Red Comms Force Net.sqd 


Comms Dest 
Sqd(s) 1-way 


Specifies which squads it is 
communicating with. This is a 
one-way communication mode 
only, i.e., the squad will only 
send messages to the specified 
squad, it will not receive any 
messages from the destination 
squad. 


Comms Dest 
Sqd(s) 2-way 


Specifies which squads it is 
communicating with. This is a 
two-way communication mode 
only, i.e., the squad will only 
send messages to the specified 
squad, it will not receive any 
messages from the destination 
squad. 


The destination squad is 
referenced by the squad index. 
The formatting of the input is 
by the use of comma separated 
values (csv). Hence, if the 
destination squads are 3, 4 and 
5, the input would be 3,4,5. 


Run Start Delay 


Specifies the time for the Run 
Start state of the MANA 4.0. 
This allows the user to decide 
the time at which the squad is 
activated to start moving within 
the scenario. 


For this option to be applicable, 
all the generic squad files that 
are modeled will need to have 
the Run Start state selected. 


Table 9. RSG Tool Scenario Generation User Interface Column Description 
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If there is a need to increase the number of row entries, the entire row should be 
copied and paste to create the next row. The reason is that there are hidden columns 
within the excel sheet that references the paths of the corresponding import files for 
generation for the MAN A scenario file. The failure to do so will result in an error. 


After all the squads that have to be generated are ready, the next step is to hit on 
the “Generate” button. This will set off a series of prompts to the user to create his 
scenario file. The prompts are described as follows. 


Figure 48 shows the first prompt to the user to select the directory to save the 
individual squad files that will be generated. 



Figure 48. Generation of MANA Model File Prompt 1 

Figure 49 shows the second prompt requires the user to specify the base scenario 
file that was previously created. 
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Figure 49. Generation of MANA Model File Prompt 2 


Figure 50 shows the third prompt requires the user to specify where he would like 
the final scenario file to be generated for subsequent retrieval. 


I Ii-VAT - I 


i r 5 r 3,*,i,6 


I*** 


t.sqd r 


Select Directory to Save 


i^y 


■ Select Directory to uve Scenario XML file 


0K I 


*” r . 


Figure 50. Generation of MANA Model File Prompt 3 


Once the scenario file is successfully generated, a final pop-up message, as shown 
in Figure 51, will come up informing the user of the successful generation of the scenario 
file. The user can then proceed to retrieve the file from the directory that was 
previously specified. 
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Figure 51. Generation of MANA Model File Successful Generation Prompt 

C. RAPID DESIGN OF EXPERIMENTS (RDOE) INTERFACE 

A large scenario will also require the ability to rapidly create design of 
experiments. The study files will require the user to specify the characteristics of the 
squads that are to be varied in accordance with the design of experiments. This tool will 
allow the user to create large study files in fractions of a second. The interface and how 
the study file can be generated are described here. A screenshot of the interface is as 
shown in Figure 52. 
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Figure 52. Screenshot of the Rapid Design of Experiments Tool 
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The first step of the rapid design of experiments tool is to set up an empty study 
file so that the study paths can be filled. This is achieved using the XStudy tool that was 
developed by Steve Upton, Research Associate at the SEED Centre. Inside this empty 
study, the user needs to specify the inputs like the name of the design of experiments csv 
file, his name, etc. The variables to be varied will not need to be specified, since they will 
be generated by the RDOE tool. 


The columns of the rapid design of experiments tool is described in Table 10. 


S/N 

Column 

Name 

Function 

Remarks 

1 . 

Variable 

Name 

Specifies the 
name of the 
variable that you 
are varying. 


2. 

Sample 

XPath 

Specifies the 
XML path in 
which the 
variables will be 
varied. 

This can be obtained from XStudy 1.0 by checking the 
path that will be varied. A sample of input is 
/specification/Squad[19]/state[l]/range[4]/RangeVal 
This path essentially changes the number of hits a 
particular agent can sustain before being killed. 

3. 

Squad 

Type 

Specifies the 
generic squad 
type. 

The input for this must correspond with that found in 
the column for the generic squad import column of the 
rapid scenario generator tool. A sample input will look 
this: BLUE_IFV.sqd 

4. 

Squad 

Numbers 

Specifies the 
squad index that 
is relevant to the 
squads whose 
parameters will 
be varied. 

This column is highlighted in Red and should not be 
modified by the user at all. The background VBA 
routine will generate the inputs required for this 
column. 

5. 

Squad 

Range 

Specifies the 
squad indexes 
that will be 
searched for the 
relevant squad 
type. 

This specifies the squad ranges to look at. For 
instance, you may want to vary the characteristics for a 
certain group of tanks but the others. This feature will 
allow you to do that. 


Table 10. Rapid Design of Experiments (RDOE) Tool 


Once all that is done, click on the “Create Study File” button and the user will be 
prompted to select the empty study file. The caption is named “Select MANA XML base 
scenario file.” The screenshot of the prompt is as shown in Figure 53. 
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Figure 53. Generation of Empty Study File 


Once the user selects his empty study file, the study file will be created in a very 
short period time with the following confirmation shown in Figure 54. 




DOE File Generated Successfully 


laSat 





Figure 54. Confirmation on Successful Generation of DOE File 


The empty study file will then be over-written with the additional information 
from the DOE file. The user can then extract the file from that location and submit it 
for runs on the cluster. 

D. GENERATION OF INDIVIDUAL EXCURSION FILES 

This section describes how the user can generate individual excursion files from 
the design of experiments file (which consists of all the design points). This allows the 
user to be able to play back individual files, and observe how the simulation plays out 
visually, when there are unusual or unexpected findings in the output data that can only 
be confirmed and better understood if it is actually seen. 
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In addition, this also provides users the option to run the individual design points 
from their own machines if a cluster is not readily available. 

This tool is particularly flexible as it does not require it to be a MANA model. As 
long as the model defined using an XML file, it will be able to generate individual files. 
This tool requires three types of files: the DOE csv file, the Study file in Old McData 
format, and the model scenario XML file. 

Figure 55shows the layout of the Excursion File Generator. It is located in a 
“Create Excursion File” sheet with only one button. By clicking on the Generate 
Excursion File button, the user will be guided through a series of steps to generate the 
excursion files. 



Figure 55. Screenshot of the Layout of the Excursion File Generator. 

As shown in Figure 56 the first prompt involves the user specifying the design of 
experiments file. 
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Figure 56. Excursion File Generator Prompting for Design of Experiment File 


Figure 57 shows the prompt request for the second file, which involves the 
specification of the model scenario file. This is where the values are to be varied in 
accordance to the design of experiments file. 
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Figure 57. Excursion File Generator Prompting for Scenario File 

The third prompt, shown in Figure 58 requires the user to specify the location of 
the Study file. 
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Figure 58. Excursion File Generator Prompting for XStudy File 


The final prompt, shown in Figure 59 requires the user to specify where the 
excursion files will be saved. After this step, the excursion files will be generated and the 
user can proceed to the specified directory to retrieve the required files. 
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Figure 59. Excursion File Generator Prompting for Location to Save All Excursion 

Files 
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APPENDIX B. DETAILED DESCRIPTION OF 
MODEL COMPONENTS 


This appendix explains in detail the behavior of the various generic squad types 
and how they interact in relation with one another. State transition diagrams detailing 
how agents behave and the event that triggers state changes is the primary mechanism to 
achieve this. It covers how the various systems behave. For instance, the obstacles’ 
effects, river crossing, and the reserve activation subcomponents are covered in detail. 
There are three main parts to this appendix: the modeling obstacles, the reserve activation 
system, and the river crossing system. 

A. OBSTACLES 

This section covers how obstacles are modeled. MANA does not have the ability 
to model obstacles explicitly. Some modeling techniques exploiting the various features 
of MANA have to be employed. In this instance, we use the refueling feature to model 
obstacles. All agents that are subject to the effects of obstacles need to have the “Refuel 
by En Type 1” trigger state enabled. As described in Chapter II, three types of agents are 
classified in this model: 

• Threat 1 - Obstacles 

• Threat 2 - Troops/Soldiers 

• Threat 3 - Vehicles (includes soft-skinned and armored vehicles) 

In this instance, we will set only vehicles to be subject to the influence of 
obstacles, since the primary purpose of obstacles is to stop vehicles and not dismounted 
troops. This means that only vehicular agents have the “Refuel by En Type 1” trigger 
state enabled, with the additional settings of (highest priority, cannot interrupt, 
interrupter) for the trigger state. For the agents subject to the effects of obstacles, upon 
being refueled by the obstacle agents, their movement speed will be zero. For these 
agents, they can have additional other states that will reflect the character and personality 
of the type of force it is representing. 
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To model obstacles that will inflict damage on agents that approach them if they 
are close enough (e.g., mines), the obstacle agent can be equipped with an additional 
weapon that will shoot the agents that approach within the “weapon” range. For the 
obstacle agents, under the “Range” settings for the obstacle agents in MANA, the 
following settings will apply: 

• Refuel Trigger Range - 5 

• Prob Refuel Enemy - 100 

Other than being stopped by obstacles, there will be a need to create agents that 
are able to clear the obstacles. In actual operational conditions, this is carried out by 
combat engineer forces. The obstacle clearing agents need not be necessarily stopped by 
the obstacle agents. If they are infantry, they will not be stopped, but if they are vehicular, 
they will be stopped. These obstacle-clearing agents will be equipped with weapons that 
will specifically target only the obstacles that they can clear; it will be useless against 
other types of obstacles. 

The state transition diagrams for the modeling of obstacles and obstacle-clearing 
agents are shown in Figure 60. 


Obstacle Agent 




Figure 60. State Transition Diagram for Modeling Obstacles Effects 
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B. RESERVE ACTIVATION SYSTEM 


This section covers how reserves are activated within the model. The main 
purpose of modeling the reserves’ behavior is to model the effect of forces with the 
flexibility to be deployed to areas where they are most required. In this thesis, there are 
sets of reserves that will be deployed at two different times, depending upon the triggers 
that will activate. The key idea behind modeling this is first to create n sets of forces with 
respect to n routes to be taken. For instance, if a platoon-sized reserve may be activated to 
reinforce via one of the three sets of routes, then there is a need to create three sets of 
platoon-sized reserves. 

With that, create three holder agents that will each hold one of the platoon-sized 
reserves. These holder agents are modeled as not visible to any agent and invulnerable. 
There will be a trigger agent that is located on the battlefield that provides information on 
enemy forces. If it senses enemy forces coming in that direction; it will send information 
to the holder agent to activate the reserves. The holder agent that is first activated to 
release the reserves will “refuel” the other holder agents and prevent any subsequent 
release of additional reserves. We will take a look at the state transition diagrams in 
Figure 61 for activating reserves along either of two possible routes in two pieces. 
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Figure 61. State Transition Diagram for Reserves Activation System 
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C. RIVER CROSSING SYSTEM 

This section covers on how the river crossing is modeled. One of the challenges in 
modeling of the river crossing is the modeling of the process of vehicles loading and 
unloading onto rafts. Vehicles will not be able to cross the river when there are no rafts. 


The agents that will need to be created and their purpose are listed in Table 11. 


S/N 

Agent Type 

Purpose 

Remarks 

1 . 

Beach strip 
trigger agent 

Upon capture of the beachstrip (own forces 
reach within sensor range), the agent indicates 
that it is captured by changing state to a 
“neutral” agent. 

Communications 
links with ERP 
regulator agent and 
Raft agent(s) 
Allegiance = 0 
(neutral agent) 
Armor =100 

2. 

Beach strip 
comms agent 

Co-locates with the beach strip trigger agent, 
sends information on the beach strip trigger 
agent to the raft agents and also the ERP 
regulating agent. 


3. 

Crossing site 
stopper agent 

Stops the vehicles from crossing the river 
when there is no raft available to bring the 
vehicles across. 

Communications 
link with ERP 
regulator agent 

4. 

Crossing site 
trigger info 
agent 

This agent provides information to the ERP 
regulating agent when there is an available 
raft. Once an available raft reaches the site, it 
will be refueled by the raft and it turns to state 
“Neutral.” This state transition will be sensed 
by the co-located Crossing site stopper agent 
which transmits the information to the ERP 
regulating agent to release vehicles to the site 
to cross the river. 


5. 

ERP regulating 
agent 

Regulates the number of vehicles that can 
“board” the raft each time, this is located at 
the assembly area at a distance from the 
crossing site. This is regulated by a weapon 
that “shoots” at vehicles waiting at the 
assembly area. It has a limited number of 
shots and reloads to release more vehicles 
when the next raft arrives. 


6. 

ERP stopper 
agent 

Stops the agent just before it is allowed to 
move to the crossing site along the river. 


7. 

Raft agent 

The raft agent acts as the “key” to opening the 
crossing site for crossing. Upon reaching the 
crossing site, it will “refuel” the ERP stopper 
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S/N 

Agent Type 

Purpose 

Remarks 



agent and it allows the vehicles waiting to 
cross to move through it. 


8. 

Troop carrier 
agent 

This is a surrogate agent that holds the actual 
vehicle that is crossing the river. This can only 
be done in MANA 4 since the embuss feature 
is only available in MANA 4. Once this 
surrogate agent crosses the river, it releases 
the vehicle it is holding, for the combat 
mission to continue. One surrogate agent is 
required for every vehicle that is crossing the 
river. 



Table 11. Description of MANA Agents for River Crossing System. 


The state transition diagram for the entire model is shown in Figure 62 and 63. 


Troop Carrier 
Agent 


ERP regulator 



Figure 62. State Transition Diagram for River Crossing Part One 
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RAFT Agent 

J,agp 



Figure 63. State Transition Diagram for River Crossing Part Two 
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APPENDIX C. OUTPUT OF CLUSTER AND 
OUTLIER ANALYSIS 


This appendix shows the detailed output results of COADM for both the urban 
and river crossing scenarios. 

A. URBAN OPERATIONS (UO) 

Figures 64 and 65 show the U matrix and the Hit map, respectively. Figure 66 
shows the locations of the outliers in the respective clusters. Table 12 gives the detailed 
statistics of the respective clusters. 
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Figure 64. UO - U-matrix and Component Planes 
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Figure 65. UO - U-matrix and Hit Map 



Figure 66. UO - Clusters and Outliers 
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1 

MeanFER 0.832 +/- 0.255 

2 

MeanFER 1.012 +/- 0.343 

MeanBlueCas 125.000 +/- 38.559 

MeanBlueCas 121.300 +/- 42.943 

MeanRedCas 161.400 +/- 6.661 

MeanRedCas 131.500 +/- 32.827 

B_Ph_AntiTank 6328.000 +/- 2913.333 

R_ArtyFireRate 117.000 +/- 54.482 

B_ArtyRadius 29.000 + /- 13.461 

Concealment 0.680 + /- 0.288 

B_ArtyFireRate 114.000 +/- 61.646 

B_ObstClearRate 57.000 +/- 25.375 

R_ArtyFireRate 109.000 + /- 66.060 

R_ResTime 1034.000 +/- 467.713 

R_ArtyGuns 4.000 + /- 1.569 

B_Ph_AntiTank 5899.000 +/- 2642.103 

B_Armrhits 6.000 +/- 2.524 

Cover 0.650 +/- 0.224 

R_Ph_SmallArms 5274.000 +/- 2853.685 

B_Armrhits 6.000 +/- 2.338 

R_CommsLat 464.000 +/- 244.014 

B_IntelSensorLat 492.000 +/- 318.668 

R_Stealth 50.000 +/- 31.233 

B_ArtyFireRate 109.000 +/- 65.279 

R_Speed 103.000 + /- 55.992 

R_CommsLat 496.000 +/- 310.476 

R_Ph_Arty 5079.000 +/- 2808.235 

R_Ph_SmallArms 5352.000 +/- 2612.081 

B_Stealth 50.000 + /- 33.217 

R_ArtyRadius 27.000 +/- 14.624 

B_Ph_Arty 4922.000 +/- 2675.858 

R_Ph_Arty 5274.000 +/- 2238.012 

B_Ph_SmallArms 4883.000 + /- 2641.498 

R_Speed 103.000 +/- 66.305 

R_ResTime 872.000 +/- 579.251 

B_Ph_SmallArms 5040.000 +/- 3147.091 

B_ResTime 886.000 +/- 517.062 

B_Speed 96.000 +/- 64.423 

B_ObstClearRate 48.000 + /- 32.154 

B_Comms_Lat 450.000 +/- 250.890 

B_ArtyGuns 6.000 +/- 2.926 

B_Ph_Arty 4961.000 +/- 2116.907 

R_Armrhits 5.000 +/- 2.602 

B_ArtyGuns 7.000 +/- 2.422 

CiviliansDensity 277.000 + /- 187.103 

CiviliansDensity 281.000 +/- 130.083 

Concealment 0.470 + /- 0.254 

B_Stealth 47.000 +/- 23.981 

B_Speed 78.000 + /- 52.775 

R_Armrhits 5.000 +/- 2.787 

R_ArtyRadius 23.000 +/- 14.842 

B_ResTime 809.000 +/- 446.499 

B_Comms_Lat 387.000 + /- 256.636 

R_Ph_AntiTank 4376.000 +/- 2906.742 

R_Ph_AntiTank 4337.000 + /- 2816.202 

R_ArtyGuns 3.000 +/- 1.169 

Cover 0.380 + /- 0.213 

R_Stealth 37.000 +/- 8.329 

B_IntelSensorLat 260.000 + /- 182.875 

B_ArtyRadius 21.000 +/- 6.623 

3 

MeanFER 1.531 +/- 0.663 

4 

MeanFER 2.120 +/- 1.909 

MeanBlueCas 174.900 +/- 46.947 

MeanBlueCas 197.200 +/- 35.031 

MeanRedCas 105.300 +/- 26.261 

MeanRedCas 95.700 +/- 27.305 

R_Ph_AntiTank 5977.000 +/- 2358.601 

B_IntelSensorLat 591.000 +/- 241.688 

B_ResTime 1083.000 +/- 339.192 

R_ArtyRadius 29.000 +/- 12.079 

Cover 0.700 + /- 0.248 

B_ArtyRadius 28.000 +/- 12.923 

R_Ph_SmallArms 5938.000 +/- 2831.438 

Cover 0.660 +/- 0.232 

B_IntelSensorLat 545.000 + /- 155.665 

R_Armrhits 6.000 +/- 3.127 

R_Stealth 56.000 +/- 30.828 

B_ResTime 998.000 +/- 490.703 

R_ArtyGuns 4.000 +/- 1.649 

R_Speed 110.000 +/- 70.701 

R_ArtyRadius 28.000 + /- 14.516 

Concealment 0.600 +/- 0.300 

B_Speed 109.000 +/- 49.056 

R_ArtyFireRate 104.000 +/- 48.939 

Concealment 0.590 + /- 0.283 

B_ArtyGuns 7.000 +/- 2.759 

CiviliansDensity 302.000 + /- 172.709 

R_ArtyGuns 4.000 +/- 1.352 

B_ArtyGuns 7.000 +/- 3.457 

R_Ph_AntiTank 5157.000 +/- 3137.316 

R_CommsLat 454.000 +/- 247.937 

R_Stealth 50.000 +/- 25.965 

B_ObstClearRate 50.000 +/- 27.289 

B_Ph_Arty 5118.000 +/- 3474.217 

B_Ph_Arty 4766.000 +/- 3196.673 

CiviliansDensity 288.000 +/- 177.038 

B_Ph_AntiTank 4805.000 +/- 3382.685 

R_Ph_SmallArms 4883.000 +/- 2554.980 

B_ArtyFireRate 95.000 + /- 58.169 

R_ResTime 872.000 +/- 530.532 

B_ArtyRadius 24.000 + /- 16.340 

B_Stealth 49.000 +/- 24.013 

B_Armrhits 5.000 + /- 1.965 

B_Ph_SmallArms 4805.000 +/- 2863.704 

R_Ph_Arty 4649.000 +/- 2725.210 

B_Comms_Lat 436.000 +/- 230.413 

R_Armrhits 5.000 +/- 2.304 

R_Ph_Arty 4571.000 +/- 2558.947 

R_Speed 90.000 +/- 60.677 

B_Speed 79.000 +/- 52.982 

B_Stealth 45.000 +/- 27.825 

B_ArtyFireRate 89.000 +/- 46.475 

R_ResTime 745.000 +/- 566.458 

B_Armrhits 4.000 +/- 2.935 

B_Comms_Lat 380.000 +/- 230.418 

B_Ph_AntiTank 4102.000 +/- 2181.052 

R_ArtyFireRate 79.000 + /- 45.473 

B_ObstClearRate 38.000 +/- 16.942 

B_Ph_SmallArms 3829.000 +/- 3158.259 

R_CommsLat 295.000 +/- 291.025 
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5 

MeanFER 1.405 +/- 0.203 

MeanBlueCas 200.700 +/- 12.434 

MeanRedCas 143.300 +/- 12.322 
B_Ph_SmallArms 5977.000 +/- 2735.220 
R_ArtyGuns 4.000 +/- 1.620 

R_Ph_AntiTank 5704.000 +/- 2190.233 
B_ArtyGuns 8.000 +/- 3.098 

B_ArtyRadius 28.000 + /- 12.953 

R_Stealth 57.000 +/- 25.414 

B_Speed 116.000 +/- 48.452 

B_Armrhits 6.000 + /- 2.104 

B_Comms_Lat 496.000 +/- 255.388 

B_Stealth 54.000 + /- 25.667 

R_Armrhits 6.000 +/- 2.642 

R_ArtyRadius 26.000 + /- 15.215 

R_CommsLat 468.000 +/- 261.447 
B_IntelSensorLat 439.000 + /- 254.382 
R_Ph_SmallArms 5118.000 +/- 3000.531 
B_ObstClearRate 48.000 + /- 26.399 
R_ArtyFireRate 92.000 +/- 56.971 
R_ResTime 844.000 +/- 493.297 

R_Ph_Arty 4688.000 +/- 3141.539 

R_Speed 96.000 +/- 55.171 

B_ResTime 865.000 + /- 462.445 
B_ArtyFireRate 92.000 +/- 61.443 

Cover 0.400 + /- 0.284 

CiviliansDensity 251.000 + /- 152.846 
B_Ph_Arty 4102.000 + /- 2619.401 
Concealment 0.390 +/- 0.235 

B_Ph_AntiTank 4141.000 + /- 2771.929 

6 

MeanFER 1.286 +/- 0.188 

MeanBlueCas 178.700 +/- 21.082 

MeanRedCas 140.700 +/- 12.971 

B_Armrhits 7.000 + /- 2.305 

B_ArtyFireRate 121.000 + /- 54.500 
B_Comms_Lat 517.000 +/- 243.340 
R_ArtyRadius 28.000 +/- 13.346 
B_IntelSensorLat 527.000 +/- 217.520 
R_Ph_SmallArms 5782.000 +/- 2440.916 
Concealment 0.620 +/- 0.210 

R_Speed 110.000 +/- 49.799 

R_CommsLat 503.000 +/- 228.868 

B_Speed 105.000 +/- 52.962 
B_ObstClearRate 52.000 +/- 32.567 
CiviliansDensity 298.000 +/- 171.347 
R_Armrhits 6.000 +/- 2.508 

B_Ph_SmallArms 4883.000 +/- 3043.367 
B_Ph_AntiTank 4883.000 +/- 3029.953 
R_ArtyGuns 3.000 +/- 1.593 

B_ArtyGuns 6.000 +/- 3.000 

Cover 0.480 +/- 0.218 

R_Ph_Arty 4649.000 +/- 3108.667 

R_Stealth 43.000 +/- 32.496 
R_ArtyFireRate 88.000 +/- 61.248 
B_ResTime 837.000 +/- 511.633 

R_Ph_AntiTank 4493.000 +/- 2991.444 
B_Ph_Arty 4063.000 +/- 2493.992 
B_ArtyRadius 22.000 +/- 13.361 

B_Stealth 41.000 +/- 30.721 

R_ResTime 689.000 +/- 431.695 

7 

MeanFER 1.771 +/- 0.983 

MeanBlueCas 218.700 +/- 14.034 

MeanRedCas 117.400 +/- 28.023 

R_Ph_AntiTank 5782.000 +/- 2976.219 
B_Speed 123.000 +/- 51.304 

R_Stealth 58.000 +/- 30.158 

R_ResTime 998.000 +/- 442.179 

R_Armrhits 6.000 + /- 2.655 
B_ObstClearRate 54.000 + /- 30.975 
B_Stealth 53.000 +/- 29.535 

B_Comms_Lat 471.000 +/- 251.044 
R_ArtyFireRate 101.000 +/- 60.324 
B_IntelSensorLat 457.000 + /- 230.050 
R_ArtyGuns 3.000 +/- 1.353 
CiviliansDensity 293.000 + /- 179.781 
R_ArtyRadius 26.000 + /- 14.528 

B_Ph_Arty 4961.000 +/- 2867.581 
B_Ph_SmallArms 4922.000 +/- 3017.874 
R_Ph_SmallArms 4922.000 +/- 2983.169 
B_ArtyFireRate 96.000 +/- 59.453 
B_ArtyRadius 24.000 +/- 14.439 

R_Ph_Arty 4610.000 +/- 2693.747 
B_ArtyGuns 6.000 +/- 3.398 

Cover 0.440 +/- 0.245 

R_CommsLat 376.000 +/- 259.824 
Concealment 0.420 + /- 0.259 

B_ResTime 802.000 +/- 570.122 

B_Ph_AntiTank 4258.000 + /- 2732.839 
R_Speed 82.000 + /- 48.650 

B_Armrhits 4.000 +/- 1.635 

8 

MeanFER 0.728 +/- 0.248 

MeanBlueCas 90.200 +/- 40.344 

MeanRedCas 133.900 +/- 33.950 

B_ArtyGuns 8.000 +/- 2.684 

R_CommsLat 580.000 +/- 244.569 

B_Ph_AntiTank 6250.000 +/- 2782.246 
B_Ph_Arty 5899.000 +/- 2857.947 
Concealment 0.650 +/- 0.300 

B_Armrhits 6.000 +/- 2.771 

Cover 0.660 +/- 0.262 

B_Stealth 52.000 +/- 23.610 
R_ArtyFireRate 98.000 +/- 53.327 
B_Ph_SmallArms 5001.000 +/- 2698.385 
CiviliansDensity 288.000 +/- 170.397 
R_ResTime 872.000 +/- 406.271 
R_Ph_SmallArms 4844.000 +/- 2766.691 
R_ArtyRadius 25.000 +/- 11.271 

R_ArtyGuns 3.000 +/- 1.215 

B_ResTime 886.000 +/- 555.814 
B_IntelSensorLat 380.000 +/- 296.459 
R_Ph_Arty 4610.000 +/- 2586.271 
B_ObstClearRate 45.000 +/- 27.400 
B_ArtyRadius 23.000 +/- 14.491 

R_Stealth 41.000 +/- 24.411 

R_Speed 89.000 +/- 59.850 

R_Ph_AntiTank 4493.000 +/- 3135.770 
B_Comms_Lat 397.000 +/- 307.091 
B_ArtyFireRate 79.000 +/- 46.171 
R_Armrhits 4.000 +/- 2.645 

B_Speed 52.000 +/- 38.116 
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10 

MeanFER 1.179 + /- 0.212 
MeanBlueCas 120.900 +/- 32.785 
MeanRedCas 101.400 +/- 19.833 
Cover 0.770 +/- 0.175 
Concealment 0.740 +/- 0.129 
B_ObstClearRate 59.000 +/- 25.849 
B_Comms_Lat 506.000 + /- 229.679 
B_ArtyFireRate 116.000 +/- 56.523 
R_ArtyGuns 4.000 + /- 1.471 
R_Armrhits 6.000 +/- 2.105 
R_Ph_Arty 5547.000 +/- 3281.420 
R_ResTime 970.000 +/- 399.604 
CiviliansDensity 307.000 +/- 157.616 
B_Armrhits 6.000 + /- 2.183 
R_Stealth 52.000 +/- 17.480 
B_Stealth 52.000 +/- 27.253 
R_CommsLat 489.000 +/- 195.909 
B_Ph_Arty 5235.000 +/- 2849.545 
B_Speed 103.000 +/- 53.294 
R_Ph_AntiTank 5118.000 + /- 3171.533 
B_IntelSensorLat 446.000 + /- 262.324 
B_Ph_AntiTank 5157.000 +/- 3081.957 
R_ArtyFireRate 96.000 +/- 55.436 
R_ArtyRadius 25.000 +/- 14.896 
R_Speed 93.000 + /- 59.032 
B_Ph_SmallArms 4454.000 +/- 2805.997 
B_ArtyGuns 6.000 +/- 3.143 
B_ResTime 823.000 + /- 581.669 
B_ArtyRadius 23.000 +/- 14.723 
R_Ph_SmallArms 4141.000 +/- 3421.567 


9 

MeanFER 1.120 +/- 0.135 
MeanBlueCas 175.400 + /- 18.341 
MeanRedCas 158.400 +/- 4.537 
R_Ph_Arty 6407.000 +/- 2363.833 
R_Speed 121.000 +/- 63.463 
B_Armrhits 7.000 +/- 2.197 
B_ResTime 1055.000 +/- 443.052 
B_ArtyRadius 27.000 +/- 14.571 
R_CommsLat 513.000 +/- 256.769 
CiviliansDensity 314.000 +/- 167.105 
B_Ph_Arty 5430.000 +/- 3025.069 
R_ArtyFireRate 106.000 +/- 57.050 
B_IntelSensorLat 492.000 + /- 228.871 
B_Speed 109.000 + /- 58.854 
B_Comms_Lat 478.000 + /- 280.959 
B_Ph_SmallArms 5235.000 +/- 2948.356 
R_ResTime 928.000 + /- 557.643 
B_ArtyFireRate 102.000 +/- 52.197 
B_ObstClearRate 50.000 +/- 27.988 
R_ArtyGuns 3.000 +/- 1.377 
B_Stealth 50.000 +/- 29.616 
B_ArtyGuns 6.000 +/- 3.439 
R_Ph_SmallArms 4805.000 +/- 3079.926 
R_Stealth 46.000 +/- 24.466 
B_Ph_AntiTank 4844.000 +/- 2390.488 
R_ArtyRadius 25.000 +/- 14.468 
R_Armrhits 5.000 +/- 2.670 
R_Ph_AntiTank 4297.000 +/- 3043.913 
Cover 0.370 +/- 0.236 
Concealment 0.350 +/- 0.221 


Table 12. UO - Cluster Data 

B. RIVER CROSSING OPERATIONS (RCO) 

Figures 67 and 68 show the U matrix and the Hit map, respectively. Figure 69 
shows the locations of the outliers in the respective clusters. Table 13 gives the detailed 
statistics of the respective clusters. 
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Figure 67. RCO - U-Matrix and Component Planes 



Figure 68. RCO - U-Matrix and Hit Map 
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Figure 69. RCO - Cluster and Outlier Map 
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1 

BlueCas 251.000 +/- 23.610 

2 

BlueCas 85.000 +/- 45.548 

RedCas 241.000 +/- 17.107 

RedCas 243.000 +/- 51.898 

FER 1.052 +/- 0.087 

FER 0.323 +/- 0.192 

R_ArtyRadius 32.000 +/- 11.479 

R_Ph_Arty 6016.000 + /- 2494.370 

R_Speed 117.000 + /- 52.027 

Cover 0.630 +/- 0.237 

B_Ph_Arty 5977.000 +/- 2786.642 

R_ResTime 991.000 +/- 466.200 

B_IntelSensorLat 520.000 +/- 205.654 

R_ArtyGuns 7.000 + /- 3.044 

B_Armrhits 7.000 +/- 2.063 

R_Stealth 54.000 +/- 27.658 

R_Ph_Arty 5860.000 +/- 2799.317 

R_IntelSensorLat 468.000 + /- 266.140 

B_ObstClearRate 61.000 +/- 30.493 

R_Ph_SmallArms 5274.000 +/- 2483.352 

R_Armrhits 6.000 + /- 2.315 

B_Ph_AntiTank 5586.000 +/- 2778.503 

R_CommsLat 506.000 + /- 282.144 

B_Ph_Arty 5313.000 +/- 2330.153 

B_Stealth 56.000 +/- 28.108 

Concealment 0.520 +/- 0.301 

Cover 0.540 + /- 0.254 

B_ArtyRadius 26.000 +/- 16.411 

R_ArtyFireRate 105.000 +/- 51.325 

R_CommsLat 464.000 +/- 243.492 

B_Comms_Lat 485.000 +/- 243.577 

B_Ph_SmallArms 5118.000 + /- 2921.311 

B_Speed 104.000 +/- 52.464 

R_ArtyRadius 26.000 +/- 14.933 

R_Stealth 48.000 +/- 29.986 

B_Armrhits 5.000 +/- 2.826 

R_ResTime 879.000 +/- 572.881 

R_Speed 98.000 + /- 59.484 

B_Ph_AntiTank 4688.000 +/- 2956.785 

B_ArtyGuns 11.000 +/- 7.154 

B_ArtyFireRate 98.000 +/- 62.075 

R_Ph_AntiTank 4532.000 + /- 3009.575 

B_ArtyGuns 11.000 +/- 6.880 

B_Comms_Lat 432.000 +/- 260.874 

R_ArtyGuns 6.000 +/- 3.342 

B_ArtyFireRate 95.000 +/- 59.160 

R_Ph_AntiTank 4415.000 +/- 2709.089 

R_ArtyFireRate 87.000 +/- 54.806 

Concealment 0.420 +/- 0.280 

R_Armrhits 5.000 +/- 2.810 

B_ArtyRadius 21.000 +/- 11.509 

B_ObstClearRate 42.000 + /- 22.948 

R_Ph_SmallArms 4376.000 +/- 2845.341 

B_Speed 88.000 + /- 60.030 

R_IntelSensorLat 362.000 +/- 257.328 

B_IntelSensorLat 225.000 + /- 210.997 

B_Ph_Sma11Arms 3985.000 +/- 2393.248 

B_Stealth 35.000 + /- 23.089 

3 

BlueCas 275.000 +/- 34.399 

4 

BlueCas 212.000 +/- 63.048 

RedCas 246.000 +/- 9.817 

RedCas 240.000 +/- 14.962 

FER 1.138 +/- 0.137 

FER 0.906 +/- 0.250 

B_IntelSensorLat 612.000 +/- 195.587 

R_Ph_AntiTank 6250.000 +/- 2227.190 

R_Armrhits 7.000 +/- 2.293 

Cover 0.640 +/- 0.264 

R_Stealth 59.000 +/- 29.121 

Concealment 0.580 +/- 0.217 

B_ArtyRadius 29.000 +/- 15.080 

B_IntelSensorLat 468.000 +/- 249.544 

B_Ph_Sma11Arms 5469.000 + /- 2836.027 

B_ArtyGuns 13.000 +/- 6.592 

B_Ph_Arty 5547.000 +/- 2490.606 

R_Armrhits 6.000 +/- 2.569 

R_ResTime 956.000 +/- 538.406 

R_ResTime 942.000 + /- 545.644 

R_Speed 108.000 +/- 48.788 

B_Comms_Lat 492.000 +/- 271.703 

B_ArtyGuns 13.000 +/- 7.059 

R_CommsLat 464.000 +/- 253.441 

R_Ph_Arty 5313.000 +/- 2777.764 

R_Ph_SmallArms 5118.000 +/- 2678.158 

R_ArtyGuns 7.000 +/- 3.744 

B_Ph_AntiTank 5313.000 +/- 2637.410 

B_ArtyFireRate 105.000 + /- 60.292 

B_Ph_SmallArms 5079.000 + /- 2982.239 

R_ArtyFireRate 102.000 +/- 63.415 

B_Armrhits 6.000 + /- 2.568 

B_Ph_AntiTank 5157.000 +/- 2975.745 

R_IntelSensorLat 415.000 +/- 244.891 

R_Ph_AntiTank 4805.000 +/- 3167.771 

B_Speed 100.000 +/- 60.273 

R_CommsLat 411.000 +/- 233.826 

B_ObstClearRate 50.000 +/- 28.349 

B_Stealth 49.000 + /- 28.234 

R_ArtyGuns 6.000 +/- 3.562 

B_Armrhits 5.000 +/- 1.881 

B_ArtyRadius 22.000 +/- 12.912 

B_Speed 98.000 +/- 59.438 

B_Ph_Arty 4297.000 +/- 2571.069 

B_Comms_Lat 443.000 +/- 236.218 

R_Speed 91.000 +/- 58.299 

R_Ph_SmallArms 4610.000 +/- 3274.895 

B_ArtyFireRate 95.000 +/- 55.559 

B_ObstClearRate 48.000 +/- 28.181 

R_ArtyFireRate 87.000 + /- 59.027 

Cover 0.420 +/- 0.254 

R_ArtyRadius 23.000 +/- 14.166 

R_IntelSensorLat 376.000 +/- 257.706 

R_Stealth 40.000 + /- 29.397 

R_ArtyRadius 23.000 +/- 14.187 

R_Ph_Arty 4141.000 +/- 2391.200 

Concealment 0.360 +/- 0.237 

B_Stealth 35.000 +/- 26.943 

5 

BlueCas 133.000 +/- 58.566 

6 

BlueCas 227.000 +/- 29.267 

RedCas 249.000 + /- 7.086 

RedCas 247.000 +/- 10.170 

FER 0.528 +/- 0.238 

FER 0.944 +/- 0.125 

B_Stealth 69.000 + /- 26.951 

R_ArtyFireRate 131.000 +/- 40.942 

B_Ph_AntiTank 6446.000 +/- 2262.377 

R_IntelSensorLat 545.000 +/- 233.772 
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B_ArtyFireRate 116.000 +/- 57.018 

R_CommsLat 559.000 +/- 206.622 

B_ArtyGuns 14.000 +/- 6.506 

R_Ph_SmallArms 5704.000 +/- 2744.418 

B_Armrhits 6.000 +/- 2.849 

B_Stealth 60.000 +/- 24.246 

R_CommsLat 503.000 +/- 237.146 

B_Speed 114.000 +/- 54.811 

B_ArtyRadius 27.000 +/- 13.873 

B_Armrhits 6.000 + /- 2.486 

Concealment 0.520 +/- 0.202 

R_ArtyRadius 28.000 + /- 14.974 

R_Stealth 51.000 +/- 29.995 

B_ObstClearRate 57.000 + /- 27.294 

B_IntelSensorLat 422.000 +/- 304.042 

R_Armrhits 6.000 +/- 2.550 

B_Ph_Arty 4961.000 + /- 2292.559 

B_Ph_AntiTank 5508.000 +/- 3014.905 

R_IntelSensorLat 432.000 +/- 221.962 

R_ResTime 942.000 + /- 515.091 

B_Speed 104.000 +/- 65.679 

B_ArtyFireRate 106.000 +/- 55.926 

B_ObstClearRate 53.000 +/- 31.527 

R_ArtyGuns 7.000 +/- 3.076 

R_Ph_Arty 4922.000 +/- 2873.673 

R_Stealth 50.000 +/- 25.879 

R_ArtyFireRate 95.000 +/- 58.763 

B_Ph_SmallArms 5001.000 +/- 2899.274 

B_Ph_Sma11Arms 4649.000 +/- 3062.647 

B_ArtyGuns 12.000 +/- 6.736 

R_Ph_AntiTank 4532.000 +/- 3325.646 

B_Comms_Lat 471.000 +/- 247.789 

R_ArtyGuns 6.000 +/- 3.092 

B_Ph_Arty 4805.000 +/- 3264.273 

R_Ph_SmallArms 4376.000 +/- 2400.096 

Cover 0.480 +/- 0.299 

R_ResTime 823.000 +/- 569.757 

B_IntelSensorLat 355.000 +/- 242.410 

B_Comms_Lat 383.000 +/- 287.544 

R_Speed 92.000 +/- 59.747 

R_ArtyRadius 21.000 +/- 13.372 

B_ArtyRadius 22.000 +/- 12.992 

R_Armrhits 4.000 +/- 2.579 

R_Ph_Arty 4532.000 + /- 2814.683 

Cover 0.260 +/- 0.181 

Concealment 0.370 +/- 0.226 

R_Speed 72.000 +/- 39.067 

R_Ph_AntiTank 3633.000 +/- 2497.540 

7 

BlueCas 254.000 +/- 42.210 

8 

BlueCas 139.000 +/- 67.397 

RedCas 209.000 +/- 49.309 

RedCas 250.000 +/- 12.683 

FER 1.223 +/- 2.067 

FER 0.548 +/- 0.273 

B_ArtyRadius 29.000 +/- 13.299 

B_Ph_AntiTank 6641.000 +/- 2645.964 

R_Speed 113.000 +/- 56.263 

B_ArtyFireRate 119.000 +/- 53.514 

Cover 0.590 +/- 0.283 

R_ArtyGuns 7.000 + /- 2.859 

B_Ph_Sma11Arms 5391.000 +/- 2657.256 

R_ArtyRadius 30.000 +/- 11.587 

R_Ph_SmallArms 5352.000 +/- 2950.857 

B_Speed 119.000 +/- 49.704 

B_ObstClearRate 59.000 +/- 30.402 

B_Ph_SmallArms 5586.000 +/- 2967.950 

Concealment 0.540 +/- 0.294 

B_ArtyGuns 14.000 +/- 6.231 

B_IntelSensorLat 464.000 +/- 226.833 

R_Stealth 54.000 +/- 29.570 

R_Ph_AntiTank 4883.000 +/- 2807.337 

B_Armrhits 6.000 +/- 2.238 

R_Stealth 49.000 +/- 28.894 

B_ArtyRadius 26.000 + /- 15.628 

B_Ph_Arty 4922.000 +/- 2934.563 

B_Comms_Lat 499.000 +/- 269.247 

R_Ph_Arty 5118.000 +/- 2965.816 

R_IntelSensorLat 450.000 + /- 261.830 

R_ArtyGuns 6.000 +/- 2.870 

R_Ph_Arty 5235.000 +/- 3449.621 

R_Armrhits 5.000 +/- 2.503 

B_Stealth 53.000 +/- 27.912 

B_ArtyFireRate 101.000 +/- 58.012 

Concealment 0.480 +/- 0.282 

B_Stealth 50.000 +/- 27.135 

R_CommsLat 425.000 +/- 254.235 

R_IntelSensorLat 408.000 +/- 269.562 

R_Ph_SmallArms 4844.000 +/- 3048.396 

B_Comms_Lat 446.000 +/- 262.682 

R_ResTime 865.000 +/- 506.741 

R_ResTime 844.000 +/- 544.525 

R_Ph_AntiTank 4454.000 +/- 3410.422 

B_Speed 93.000 +/- 53.598 

R_ArtyFireRate 88.000 + /- 40.152 

B_ArtyGuns 11.000 +/- 6.376 

R_Armrhits 5.000 + /- 2.526 

R_ArtyFireRate 86.000 +/- 56.040 

B_ObstClearRate 45.000 +/- 28.114 

R_ArtyRadius 23.000 +/- 12.325 

Cover 0.370 +/- 0.245 

R_CommsLat 323.000 +/- 257.642 

B_IntelSensorLat 253.000 +/- 146.142 

B_Armrhits 4.000 +/- 2.679 

B_Ph_Arty 3555.000 +/- 2482.486 

B_Ph_AntiTank 3321.000 +/- 2430.416 

R_Speed 78.000 +/- 62.549 


Table 13. RCO - Cluster Data 
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APPENDIX D. INTERNATIONAL DATA FARMING WORKSHOP 

17 

TEAM REPORT 


The thesis is presented in International Data Farming Workshop 17, held in 
Garmisch, Germany from 21 Sep 08 - 26 Sep 08. Additional work was done on exploring 
the tactics, techniques and procedures (TTPs) of the defender operating in an Urban 
Terrain. Essentially it is a manual red teaming effort to gain insights on how the defender 
should locate his static forces and how the defender should deploy his reinforcements. 


TEAM 5 MEMBERS 

CPT CHER HOWE, ONG 
Naval Postgraduate School, 
USA/Singapore 

CPT WEE CHUNG, SIM 

Republic of Singapore Navy, Singapore 

KURT GRAU 

IABG, Germany 

INTRODUCTION 

Modern warfare is characterized by the 
use of combined arms where various types 
of forces come together to fight as a single 
entity, as a system-of-systems. Typically, 
for land forces, system-of-systems are 
battalion level and above. In this 
workshop, the terrain that was selected to 
be studied is the urban terrain. The team’s 
objective was to explore the tactics, 
techniques and procedures (TTPs) of the 
defender operating in an urban terrain. The 
specific aim was to determine how the 
defender should locate his defensive 
positions and also how he should deploy 
his reserves. The scenario map was 
developed by the DSO National 
Laboratories, Singapore. 


DESCRIPTION OF SCENARIO 

The scenario examined during the 
workshop was a battalion attack of a 
company plus defended urban locality. 
The defender deploys a section and a 
platoon plus of mechanized infantry at the 
forward positions each along the 3 main 
axes. A company minus sized mechanized 
infantry reserve organized into 2 echelons 
is held at the rear to reinforce the forward 
positions. The entire defense is supported 
by a mortar platoon providing support fire. 
The green rectangles are obstacles 
emplaced in the defended locality. The 
defender’s defense is as shown in Figure 1 
below: 



Figure 1: Defense Plan 


105 






The attacker inserts a platoon sized block 
force to the depth and advances along 2 
axes with a company of mechanized 
infantry each, holding a company minus 
mechanized infantry as the reserve that 
will be activated upon breaking through 
either axes. The attack is supported by an 
artillery battery and a UAV. The attack 
plan is as shown in Figure 2. 



Figure 2: Attack Plan 


DESIGN OF EXPERIMENTS 

For the defensive positions, 3 concepts are 
studied: 

• Forward - This is where the 
defender pushes all his static forces 
to the front positions. 

• Depth - This is where the defender 
locates all his forces at the rear of 
the operating area. 

• Mixed - The defender deploys his 
forces uniformly along the axes. 

For the reinforcement routes, 3 concepts 
are studied: 

• No reserve - No reserves, all 
forces are deployed at the static 
defense positions. 


• Frontal - The reserves are 
deployed “head-on” with the 
direction of the attacker. 

• Flanking - The reserves comes 
from the flanks of the attacker. 

Figure 3 shows how the defender deploys 
his forces according to the concepts 
mentioned above: 


Static Forces - Defended Positions 
Strong Forward Strong Depth Mixed 



Mobile Forces - Reserves 

No Reserves Flanking Manuever Frontal Maneuver 



WWW.NPS.EDU 


Figure 3: Design of Experiments - Defender Concepts of 
Defense 

These various combinations of static 
forces and reserves make up for a total of 
3x3 = 9 combinations of tactics that the 
defender can employ. A total of 10 
replicates are made for each of the 
combination of static and reserve 
employment concept. 

RESULTS 

From Figure 4 below, we can see that a 
mixed deployment strategy produces the 
smallest mean Force Exchange Ratio. In 
addition, from the box plot in Figure 4, we 
can see that the variation of outcome for a 
mixed deployment strategy is the smallest 
among the 3 concepts. 
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Figure 4: Analysis for deployment of static forces 


From Figure 5 below, we can see that 
deploying reserves to the flanks of the 
attacker produces the smallest mean Force 
Exchange Ratio. In addition, the variation 
of the battle outcome for a flanking 
maneuver is also the smallest. 



Figure 6: Combined Analysis 


Employment of Reserves 
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Figure 5: Analysis for deployment of reserves. 


From Figure 6 below and together with the 
analysis done previously, we can see that 
employing a mixed deployment strategy 
and deploying the reserves to the flanks of 
the attacker is the most optimal and also 
robust combination for the defender in the 
urban terrain. 


CONCLUSION 

The results obtained from workshop 
provided interesting insights on how 
tactics and doctrine can be evaluated and 
validated using data farming techniques. 
The results suggest that for the defender, 
the best tactic would be to spread his 
forces along main axes of movement and 
have the reserves reinforce from the flanks 
of the attacker. By spreading his forces 
along the main axes, he would be able to 
reduce his vulnerability to the attacker’s 
artillery fires. In addition, by deploying his 
reserves from the flanks, the defender 
would be able to avoid a head-on fire fight 
with the attacker. The reserves would 
engage in decisive combat with the 
attacker only when they arrive at the 
position to be reinforced. 
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